Test Plan Determination#
This activity takes the results from previous activities and builds a unique test plan for each product, which stays valid as long as there are no changes impacting the organizational maturity or architecture maturity scores. If there are changes to scores during the current RABET-V iteration, the test plan determination must be performed again.
The test plan is a crosstab decision table. Artifacts from earlier activities, such as the submission review, organizational assessment, and architecture assessment serve as inputs to the table. The output of the test plan determination is a set of testing rigors to be used during product verification. A testing rigor is determined for each control family.
These testing rigors are Full, Basic, and Streamlined. The names reflect the rigor that applies to confirm the effectiveness of the control family, with Full applying the most rigor and Streamlined the least.
The chosen testing rigor for a given control family is based on the change types identified for the product’s current iteration and the organizational maturity and architecture maturity scores for the product. For instance, change types that indicate changes to security service components will require higher scores to receive Basic or Streamlined testing. Minor changes may receive less testing even with relatively lower scores.
Based on initial findings during the product verification activity, some tests may be made more rigorous than indicated in the test plan, but they cannot be made less rigorous.
The Full testing rigor is testing all the security requirements in the security control families for all the security services. This level of rigor is used on initial iterations, future iterations if the overall organizational and architecture maturity scores are too low to allow for Basic or Streamlined testing, or for certain change types.
The Basic testing rigor is used with sufficently good maturity scores from the organizational and architecture scores from the most recent assessment. This level of rigor includes the level one requirements plus the claimed requirements that represent 50% of the remaining requirements at levels two and three for each security control family that is impacted by application changes since the last iteration.
The Streamlined testing rigor is used with excellent maturity scores from the organizational and architecture scores from the most recent assessment. This level of rigor includes the level one requirements plus the claimed requirements that represent 20% of the remaining requirements at levels two and three for each security control family that is impacted by application changes since the last iteration.
Inputs#
Change Type
Organizational Maturity Score
Architecture Maturity Score
Outputs#
Product Test Plan
Workflow#
Review assessment scores#
Organizational assessment scores and architecture assessment scores serve as inputs.
The architecture maturity score for each RABET-V control family form the column headers of the table. The rows of the table list the change types. Each change type is associated with an organizational assessment score. The first change type matching any of those identified during submission review uniquely selects the applicable organizational assessment score (i.e., when more than one change type applies, the highest risk one takes precedence over the others). The organizational assessment and architecture assessment scores are then summed for each control family, resulting in scores between 0.0 and 6.0.
Determine testing rigors#
Each numeric score is converted to a testing rigor based on a predefined set of thresholds associated with the change type. These thresholds determine how high a score must be to receive a certain level of testing. For example, a product with an Operating system patch change type and a combined organizational and architecture score of 2.5 or greater will receive Streamlined testing. However, a change of Security patch of security service component(s) with the same score would receive Full testing. The test plan matrix is given below:
