This document specifies the procedures to test implementations of EVRC-A, EVRC-B, or EVRC-WB compatible variable-rate speech codecs either by meeting the bit-exact implementation, or meeting recommended minimum performance requirements. The EVRC-A is the Service Option 3 (SO 3) speech codec, the EVRC-B is the Service Option 68 (SO 68) speech codec, and the EVRC-WB is the Service Option 70 (SO 70) speech codec, all described in [1]. The procedures specified in this document for the SO 3 speech codec are fully consistent with those contained in [3]. The SO 3 speech codec is used to digitally encode the speech signal for transmission at a variable data rate of 8550, 4000, or 800 bps. The SO 68 speech codec is used to digitally encode the speech signal for transmission at a variable data rate of 8550, 4000, 2000, or 800 bps. The SO 70 speech codec is used to digitally encode the speech signal for transmission at a variable data rate of 8550, 4000, or 800 bps.

Like some other speech coding standards, this standard provides a bit-exact method of verifying the test codec for minimum performance. In this optional procedure, a given set of test vectors are input to the test codec and the output vectors from the test codec must be bit-exact with the output vectors given in the software distribution which is associated with this standard. If they are bit-exact, the test codec passes the minimum performance requirement and no further testing is required. The bit-exact mode of testing, however, is only applicable to codecs whose design conforms in all respects to the algorithmic description of the specific EVRC service option, including the noise suppression, rate determination and post-filter components.

Should the candidate EVRC differ in any of these components, the test codec shall be tested using the objective and subjective tests prescribed by this standard. That is, EVRC-compliance of a "test codec" can be achieved by either:

• Complying with Sections 2.1.1 and 2.1.2 (SO 3), or Sections 2.2.1 and 2.2.2 (SO 68), or Sections 2.3.1 and 2.3.2 (SO 70), and demonstrating bit-exactness according to the procedure described in Section 3.1.4 (SO 3), or Section 3.2.4 (SO 68), or Section 3.3.4 (SO 70), respectively.

• Following the objective and subjective testing procedures set forth in Sections 2.1.1 and 2.1.2, or Sections 2.2.1 and 2.2.2, or Sections 2.3.1 and 2.3.2 of this standard.

With the exception of Sections 3.1.4, 3.2.4, and 3.3.4, the remaining text applies only to implementations that do not satisfy the requirement for bit-exactness.

Testing the codec is based on two classes of procedures: objective tests and subjective tests. In the event that the test codec fails any of the objective or subjective tests, the test codec fails the compliance test. Objective tests are based upon actual measurements from the speech codec function. Subjective tests are based on listening tests to judge overall speech quality. The minimum subjective requirement for the test codec is based upon the ability of the test codec to demonstrate performance equivalent to or better than that of the specific EVRC floating-point bit-exact codec within a fixed allowable statistical error.

The purpose of the testing is not only to ensure adequate performance between one manufacturer's encoder and decoder but also that this level of performance is maintained with operation between any pairing of manufacturers' encoders and decoders. This interoperability issue is a serious one. Any variation in implementing the exact standard must be avoided if it cannot be ensured that minimum performance levels are met when interoperating with all other manufacturers' equipment meeting the standard. This standard provides a means for measuring performance levels while trying to ensure proper interoperation with other manufacturers' equipment.

The issue of interoperation can only be definitively answered by testing all combinations of encoder/decoder pairings. With the number of equipment manufacturers expected to supply equipment, this becomes a prohibitive task; therefore, the objective and subjective tests rely upon the use of a "master codec". The master codec is defined as the floating-point implementation of specific EVRC written in the C programming language. The master codec software which is described in Section 3.1.3 (SO 3), Section 3.2.3 (SO 68), or Section 3.3.3 (SO 70) is used as part of the interoperability testing.