Benchmarking Methodology WG (bmwg) Thursday, August 5, 2004, 1300-1500 ======================================= CHAIRS: Kevin Dubray Al Morton AGENDA: 1. Working Group Status (Chairs) Especially conterm, ospf*, dsmterm, ipsec term, benchres term, and Network Convergence Considerations drafts. Check the BMWG mail archive for comments: http://www.ietf.org/mail-archive/working-groups/bmwg/current/ 2. Revised Milestones. (Chairs) 3. IGP Data plane convergence benchmark I-Ds. Changes from 02 to 03 to address comments raised at WGLC. Were the comments adequately addressed? (S.Poretsky) http://www.ietf.org/internet-drafts/draft-ietf-bmwg-igp-dataplane-conv-term-03.txt http://www.ietf.org/internet-drafts/draft-ietf-bmwg-igp-dataplane-conv-meth-03.txt http://www.ietf.org/internet-drafts/draft-ietf-bmwg-igp-dataplane-conv-app-03.txt Chairs may also comment here on the Active Review Template Experiment in the BMWG Last Call Process. 4. Techniques for Benchmarking Core Router Accelerated Life Testing. First Look at Methodology. Changes to Term. Where is WG input sought? Issues? Comments? Concerns? (S. Poretsky et al.) http://www.ietf.cnri.reston.va.us/internet-drafts/draft-ietf-bmwg-acc-bench-term-03.txt http://www.ietf.cnri.reston.va.us/internet-drafts/draft-ietf-bmwg-acc-bench-meth-00.txt 5. New Work Proposal on Hash and Stuffing (it's not "What's for dinner?") This is a proposal for a new category of benchmarking memo: a "plug-in" guideline applicable to all benchmarking work (and not a benchmark, related term, or method of measurement). Specifically, current benchmarking methods do not require reporting of addresses used, or address the possibility of bit/byte stuffing in link-layer technologies. Addresses, payload contents, and the presence of stuffing all can affect the results. Does the WG support an activity to document these potential pitfalls? Related draft: (D. Newman and T. Player) http://www.ietf.org/internet-drafts/draft-newman-hash-stuffing-00.txt 6. Follow-up discussion of drafts that "plug-in" to other work, such as the hash-stuffing draft. Another example is Network Convergence Considerations, which plugs into live network characterization: http://www.watersprings.org/pub/id/draft-white-network-benchmark-00.txt How does this new memo category affect future BMWG work? (Chairs) 7. New Work Proposal on LDP Convergence Benchmarking With substantive progress being made on OSPF and IGP-dataplane convergence topics, BMWG could consider another convergence work item. Is there sufficient WG interest (e.g., expertise, readership, participation) to support this topic as a work item? We have the following description from the authors of http://www.ietf.org/internet-drafts/draft-eriksson-ldp-convergence-term-01.txt (T.Eriksson, S.Poretsky, R.Papneja) Motivation - The purpose of this document is to further the BMWG's efforts for convergence benchmarking by now focusing on LDP convergence. This is of particular interest to many Service Providers that are considering deployment of Fast Reroute to protect VPN traffic. These Service Providers may alternatively prefer fast LDP convergence as the solution to minimize VPN traffic loss. In response, router vendors are developing implementations to achieve fast LDP convergence Times. This industry focus is now driving the need for LDP convergence benchmarking terminology and methodology. Goals - The goal is to develop a benchmarking terminology and methodology for LDP data plane convergence. The convergence benchmarking will be performed for different types of events that trigger a change of outgoing LDP label for the DUT. The events include but are not limited to lost LDP session, lost IGP adjacency, better IGP next-hop, and changes due to local or remote interface failures. Measurements will be specified for the DUT as PE and P router. For each type of convergence event the benchmark metric will be calculated from the lost packets on the data plane during the Convergence Event. The benchmark metrics can be used when comparing different vendor implementations, for example during an evaluation phase. Existing terminology used for LDP and MPLS will be used whenever possible. In some cases suitable terms might not exist at all; in these cases an explicit goal of the work is to define and document new terms. 8. "Old" Work Proposal on Protection Switching Methodology The WG has discussed this item at several previous meetings. Generic Sub-IP and MPLS proposals can now be re-combined into one using a common terminology. What are the issues for this approach? Is there still enough interest in this proposal to generate solid review and discussion? Should BMWG take it up? (Chairs, S.Poretsky, et al.) Drafts related to this proposal: Draft on MPLS Protection Benchmarking Methodology http://www.ietf.org/internet-drafts/draft-poretsky-mpls-protection-meth-03.txt Automatic Protection Switching Benchmark Terminology http://www.ietf.org/internet-drafts/draft-kimura-protection-term-02.txt Expired, see http://www.watersprings.org/pub/id/draft-kimura-protection-term-02.txt Please help contribute to a successful meeting by reading the above I-D(s) and references *before* we meet. To offer comments on BMWG work in progress or the agenda itself, please send email to: bmwg@ietf.org Alternatively, to offer potential agenda items, please email: kdubray@juniper.net and acmorton@att.com