|
|
Network Working Group M. O'Dell
Request for Comments: 2438 UUNET Technologies
BCP: 27 H. Alvestrand
Category: Best Current Practice Maxware
B. Wijnen
IBM T. J. Watson Research
S. Bradner
Harvard University
October 1998
Advancement of MIB specifications on the IETF Standards Track
Status of this Memo
This document specifies an Internet Best Current Practices for the
Internet Community, and requests discussion and suggestions for
improvements. Distribution of this memo is unlimited.
Copyright Notice
Copyright (C) The Internet Society (1998). All Rights Reserved.
2. Abstract
The Internet Standards Process [1] requires that all IETF Standards
Track specifications must have "multiple, independent, and
interoperable implementations" before they can be advanced beyond
Proposed Standard status. This document specifies the process which
the IESG will use to determine if a MIB specification document meets
these requirements. It also discusses the rationale for this
process.
3. The Nature of the Problem
The Internet Standards Process [1] requires that for an IETF
specification to advance beyond the Proposed Standard level, at least
two genetically unrelated implementations must be shown to
interoperate correctly with all features and options. There are two
distinct reasons for this requirement.
The first reason is to verify that the text of the specification is
adequately clear and accurate. This is demonstrated by showing that
multiple implementation efforts have used the specification to
achieved interoperable implementations.
The second reason is to discourage excessive options and "feature
creep". This is accomplished by requiring interoperable
implementation of all features, including options. If an option is
not included in at least two different interoperable implementations,
it is safe to assume that it has not been deemed useful and must be
removed before the specification can advance.
In the case of a protocol specification which specifies the "bits on
the wire" exchanged by executing state machines, the notion of
"interoperability" is reasonably intuitive - the implementations must
successfully "talk to each other", exchanging "bits on the wire",
while exercising all features and options.
In the case of an SNMP Management Information Base (MIB)
specification, exactly what constitutes "interoperation" is less
obvious. This document specifies how the IESG has decided to judge
"MIB specification interoperability" in the context of the IETF
Standards Process.
There are a number of plausible interpretations of MIB specification
interoperability, many of which have merit but which have very
different costs and difficulties in realization.
The aim is to ensure that the dual goals of specification clarity and
feature evaluation have been met using an interpretation of the
concept of MIB specification interoperability that strikes a balance
between testing complexity and practicality.
4. On The Nature of MIB specifications
Compared to "state machine" protocols which focus on procedural
specifications, a MIB specification is much more data oriented. To
over-generalize, in a typical MIB specification the collection of
data type and instance specifications outnumbers inter-object
procedural or causal semantics by a significant amount.
A central issue is that a MIB specification does not stand alone; it
forms the access interface to the instrumentation underneath it.
Without the instrumentation, a MIB has form but no values. Coupled
with the large number of objects even in a simple MIB specification,
a MIB specification tends to have more of the look and feel of an API
or a dictionary than a state machine protocol.
It is important to distinguish between assessing the interoperability
of applications which may use or interact with MIBs, and the MIBs
themselves. It is fairly obvious that "black-box testing" can be
applied to such applications and that the approach enjoys a certain
maturity in the software engineering arts. A MIB specification, on
the other hand is not readily amenable to black box test plans.
5. Discussion and Recommended Process
In order to meet their obligations under the IETF Standards Process,
the Operations and Management Area Directors and the IESG must be
convinced that each MIB specification advanced to Draft Standard or
Internet Standard status is clearly written, that there are the
required multiple interoperable implementations, and that all options
have been implemented. There are multiple ways to achieve this goal.
Appendix A lists some testing approaches that could be used when
attempting to document multiple implementations.
The Full Coverage or Stimulus-Response approaches are very through,
and would increase confidence that the requirement has been met, if
applied. However, experience in real-world software engineering
makes it clear that such confidence comes at an extremely high price;
even with the most exhaustive testing, it is often not clear what
precisely has been demonstrated by such testing. We believe that
both of those standards of evidence are materially beyond what can be
reasonably accomplished in an operational sense, and achieving the
requisite semantic specifications are even more unlikely.
Therefore, the Operations and Management Area and the IESG have
adopted a more pragmatic approach to determining the suitability of a
MIB specification for advancement on the standards track beyond
Proposed Standard status. Each MIB specification suggested for
advancement must have one or more advocates who can make a convincing
argument that the MIB specification meets the multiple implementation
and feature support requirements of the IETF Standards Process. The
specific way to make the argument is left to the advocate, but will
normally include reports that basic object comparison testing has
been done.
Thus any recommendation for the advancement of a MIB specification
must be accompanied by an implementation report, as is the case with
all requests for the advancement of IETF specifications. The
implementation report must include the reasons why the IESG should
believe that there are multiple implementations of the MIB
specification in question and that the all of the MIB objects in the
specification to be advanced are supported in more than one
implementation. But note that the prime concern of the IESG will be
that the underlying reasons for the interoperable implementations are
met, i.e., that the text of the specification is clear and
unambiguous, and that features of the specification which have not
garnered support have been removed.
The implementation report will be placed on the IETF web page along
with the other pre-advancement implementation reports and will be
specifically referred to in the IETF Last-Call. As with all such
implementation reports, the determination of adequacy is made by the
Area Director(s) of the relevant IETF Area. This determination of
adequacy can be challenged during the Last-Call period.
6. Security Considerations
Some may view this policy as possibly leading to a reduction in the
level of confidence people can have in MIB specifications but the O&M
Area Directors and the IESG feel that it will adequately ensure a
reasonable evaluation of the level of clarity of MIB specifications
and to ensure that unused options can be identified and removed
before the advancement of a specification.
Good, clearly written MIB specifications can be of great assistance
in the management of the Internet and other networks and thus assist
in the reduction of some types of security threats.
8. References
[RFC2026] Bradner, S., "The Internet Standards Process --
Revision 3", BCP 9, RFC2026, October 1996.
9. Authors' Addresses
Michael D. O'Dell
UUNET Technologies, Inc.
3060 Williams Drive
Fairfax, VA 22031
Phone: +1-703-206-5890
EMail: mo@uu.net
Harald T. Alvestrand
Maxware
Pirsenteret
N-7005 Trondheim, Norway
Phone: +47-73-54-57-94
EMail: Harald.Alvestrand@maxware.no
Bert Wijnen
IBM T. J. Watson Research
Schagen 33
3461 GL Linschoten
Netherlands
Phone: +31-348-432-794
EMail: wijnen@vnet.ibm.com
Scott Bradner
Harvard University
1350 Mass. Ave.
Cambridge MA 02138
Phone: +1-617-495-3864
EMail: sob@harvard.edu
Appendix A
A. Some Testing Alternatives
The IESG debated a number of interoperability and testing models in
formulating this specification. The following list is not an
exhaustive enumeration of the alternatives, but it does capture the
major plausible models which were examined in the course of the
discussion.
A.1 Basic Object Comparison
Assume the requisite two genetically unrelated implementations of the
MIB in an SNMP agent and an SNMP management station which can do a
"MIB Dump" (extract the complete set of MIB object types and values
from the agent implementation). Extract a MIB Dump from each
implementation and compare the two dumps to verify that both provide
the complete set of mandatory and optional objects and that the
individual objects are of the correct types.
A.2 Stimulus/Response Testing
Proceed as in A.1, but in addition, comprehensively exercise the two
(network) elements containing the agent implementations to verify
that all the MIB objects reflect plausible values in operational
conditions. An even stricter interpretation would require that the
MIB objects in the two network elements track identically given the
identical stimulus. While this would test "read-only" or
"monitoring" information obtained from the underlying
instrumentation, it is important to observe that such instrumentation
is actually an *application* which uses the MIB and is not part of
the MIB itself.
A.3 Full Coverage Testing
This model extends the notion of Stimulus/Response Testing to its
logical extreme. The MIB is viewed as an API and the software
engineering notion of full coverage testing is applied to a MIB.
This involves exercising all paths through the causal semantics and
verifying that all objects change state correctly in all cases.
Again, note that much more than the MIB definition is being exercised
and evaluated.
Full Copyright Statement
Copyright (C) The Internet Society (1998). All Rights Reserved.
This document and translations of it may be copied and furnished to
others, and derivative works that comment on or otherwise explain it
or assist in its implementation may be prepared, copied, published
and distributed, in whole or in part, without restriction of any
kind, provided that the above copyright notice and this paragraph are
included on all such copies and derivative works. However, this
document itself may not be modified in any way, such as by removing
the copyright notice or references to the Internet Society or other
Internet organizations, except as needed for the purpose of
developing Internet standards in which case the procedures for
copyrights defined in the Internet Standards process must be
followed, or as required to translate it into languages other than
English.
The limited permissions granted above are perpetual and will not be
revoked by the Internet Society or its successors or assigns.
This document and the information contained herein is provided on an
"AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING
TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING
BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION
HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. |
|
