Systems & Measures
Local
Connectivity
Systems & Systems Support
OSS Interface Change Management
Business Rules
Measures
New Release Testing
clear_dot.gif (43 bytes)

New Release Testing


Definitions

New release testing is the process TCs use to test an upcoming Verizon systems release that impacts the interface and business rules between TCs and Verizon. This testing will take place after Verizon has completed its internal testing of the release.

New release testing is intended for those TCs which are currently in production with Verizon, submitting and receiving pre-order or order transactions through an application-to-application interface (i.e., EDI, CORBA). This process does not apply to the Web GUI interface.

Quality Baseline Validation Test Decks and Test Accounts

Verizon has created and will maintain standard Quality Baseline Validation Test Decks of pre-order and order transactions that will be used to test a new release. The Quality Baseline Validation Test Deck is also referred to as the Regression Test Deck. Verizon will distribute the updated regression test decks for the upcoming new release through BA Change Control two weeks prior to the start of TC testing. The regression test decks are posted to the Verizon Wholesale Web site shortly after their distribution through BA Change Control.

Verizon will run the regression test decks before the TC test period begins and at the conclusion of TC new release testing.

Verizon may also develop specific progression test scenarios for the major changes in functionality of the new release. If additional progression test scenarios are developed, they will be distributed through BA Change Control two weeks prior to the start of TC testing. The progression test scenarios will be posted to the Verizon Wholesale Web site shortly after their distribution through BA Change Control.

The progression test scenarios will also be run in the CLEC Test Environment at the same time as the Quality Baseline Validation Test Decks are run in the CLEC Test Environment. Results will be reported with those of the Quality Baseline Validation Test Decks. After new release testing is concluded, some of the release specific progression scenarios may be moved into the Quality Baseline Validation Test Decks.

For Pre-Order transactions, the test decks consist of inbound requests and corresponding outbound responses. For Order transactions, the test decks consist of the LSR, the inbound EDI request, and the outbound EDI response (e.g., confirmation).

These test deck scenarios and test deck accounts are available for TCs to use during the testing period. However, TCs are not limited to these transactions and accounts and may request additional support from Verizon to build specific test accounts in the CLEC Test Environment. Such requests must be received as part of a test plan two weeks prior to the beginning of TC testing.

TCs may also request that some of their accounts that currently reside in production be moved into the CLEC Test Environment to be available for testing. Generic TC accounts in the CLEC Test Environment may be duplicated using the identification of another TC.

Getting Ready for the New Release Testing

TCs are notified of the content of the release through the change management process. TCs should review the content of the release and determine if they want to participate in the test and what transactions they would like to submit as part of the test.

Verizon will put out an industry notification requesting TCs to identify their intent to participate in the test based on the schedule at the end of this section. At that time, Verizon will publish any changes to the schedule.

TCs wishing to participate in the test should make arrangements with the CLEC Testing coordinator.

As identified on the schedule, TCs need to submit a test plan identifying the nature and volumes of transactions they intend to submit as part of the test and when the transactions will be submitted. The test coordinator will work with the TC in determining the appropriate testing scenarios, and to ensure that the test plan will meet the TC requirements to test the new release, and if additional accounts must be added to the CLEC Test Environment database based on their test plan.

New Release Testing Process

Four weeks prior to a TC impacting release, code for that release will be loaded into the CLEC Test Environment. This code will already have gone through Quality Assurance testing by Verizon. Verizon will run the test decks through the new release system code and publish the results of this test through BA Change Control. The published results will indicate if there were any differences from what was documented in the test decks previously distributed through BA Change Control.

TCs will begin new release testing on the Monday four weeks before release implementation and may submit test transactions normally between 8:00 a.m. and 5:00 p.m. Eastern Monday through Friday. Verizon may offer extended hours, or changes to the schedule, and any such changes will be communicated through the Change Management Process. The CLEC Test Environment will be unavailable for new release testing the Friday before release implementation into the CLEC Test Environment and into production. Orders that qualify for level 5 transactions will flowthrough to the service order processor. Acknowledgements, confirmations, and (where mutually agreed) completions will be provided. Order transactions that do not flowthrough will be manually entered into the service order processor and the same responses provided.

During new release testing, the testing coordinator will be the TCs point of contact to identify and resolve problems and questions. The testing coordinator will involve other Verizon personnel as required. Verizon will maintain a log of issues during the new release test. Issues will be responded to by the next business day. The status of open issues will be published and reviewed with the TCs on status calls to be held by the TC Testing Director, or designate, every Tuesday and Friday.

On the last Monday of the TC new release testing period, a special status call will be held to identify any outstanding issues that must be fixed prior to release implementation.

Verizon will not make any changes to the CLEC Test Environment while TCs are testing the new release. Defects will be fixed each Wednesday evening during TC new release testing. Emergency fixes may be implemented at times other than Wednesday evening. The last Wednesday will be reserved for fixes that must be done prior to release implementation Any of these additional fixes will be communicated through Change Management Process. This enables the TCs an opportunity to retest before the code is migrated to production.

The escalation procedure to be used, if necessary, to resolve issues during TC new release testing is at the end of this section.

Post New Release Testing

After completion of TC testing, the code will be migrated into production. The CLEC Test Environment will continue to contain this code until the code for the next release is moved into the CLEC Test Environment.

Verizon will execute the test decks in the CLEC Test Environment at the end of the new release testing period and verify that the results match the published test decks. After the code contained in the CLEC Test Environment has been migrated to production, Verizon will run the Quality Baseline Validation Test Decks in production without changing the end state of accounts; and Verizon will document the results within 5 days. Completions will not be a part of the results obtained from production.

After each release has been moved into production, Verizon will affirm that it has used software configuration management tools to ensure that the CLEC Test Environment code was successfully moved into production.

Sample New Release Testing Schedule

Monday 8 weeks prior to release implementation Verizon will request the TC's to indicate their intent to participate in Release testing via Change Management Process

Monday 6 weeks prior to release implementation
TCs will provide an initial Release Test plan to their test coordinator

Monday 6 weeks prior to release implementation
Verizon publishes test deck scenarios developed for the release. TCs provide test plan and account requirements to Verizon.

Friday 4 weeks prior to release implementation CLEC Test Environment unavailable.
Release migrated to CLEC Test Environment. Test decks run in CLEC Test Environment.

Weekend 4 weeks prior to release implementation
Verizon publishes results of test decks run in the CLEC Test Environment.

Monday 4 weeks prior to release implementation
TC testing begins.

Tuesday/Friday each of the 4 weeks before release implementation
Status calls are held.

Wednesday each of the 4 weeks before release implementation
Code fixes are implemented after 5:00PM as needed.

Monday 1 week prior to release implementation
Special status call is held.

Thursday before release implementation
TC testing concludes.

Weekend of release implementation
Test decks run in CLEC Test Environment and results published.
Release migrated to production.
Verizon verifies code successfully migrated.

Monday
Test decks run in production and results published within 5 days.

Escalation Process

  1. Purpose. To provide processes for TCs to raise and address issues arising during quality assurance testing of new releases with Verizon.
  2. Prerequisites to Escalations. The expectation is that escalations should occur only after reasonable efforts have been made to resolve the issue with Verizon's testing team and test coordinator.
  3. Escalation by a TC during Testing.
  1. If reasonable efforts to resolve an issue with Verizon fail, an individual TC (or group of TCs acting jointly) may escalate the issue to the TC Testing Director.
  2. The escalation and the subsequent responses and replies should take the form of e-mails, with copies to the industry change control distribution. The TC will also call the TC Testing Director to provide notification that an e-mail was sent.
  3. The escalation should include the following:

An explanation of the issue.

A brief history of the steps taken to resolve the issue with Verizon's testing team and testing manager.

The desired outcome of the escalation.

The impact to the TC if the issue is not resolved.

Contact information, including name, title, phone number, and e-mail address.

  1. The Director will acknowledge receipt of an escalation. The Director will provide the escalating TC with an initial finding within 1 business day and a final response within 2 business days.
  2. The initial finding should include the following:
    - A brief explanation of the issue, if different from that provided by the escalating TC.
    A brief statement of the likely resolution of the issue.
  3. The response should include the following:
    An explanation of the issue, if different from that provided by the escalating TC.
    A description of the steps taken by Verizon to resolve the escalation.
    A proposed resolution of the issue.
  4. The escalating TC must reply to the Director's response within 2 business days, informing Verizon whether the TC intends to escalate the issue further.
  5. If the Director fails to resolve the issue to the TCs satisfaction, the TC may escalate the issue to the Vice-President - Wholesale Customer Support over the TC Testing Process.
  6. The second escalation need not repeat information, but it should describe any developments subsequent to the first escalation, including a copy of the director's response.
  7. The Vice President will provide a final response within 1 business day.
  8. The reply need not repeat information, but it should describe any developments subsequent to the first escalation.
  9. If unsatisfied with an outcome, either party can seek appropriate relief.

Sample PON Tracking Spreadsheet

TC Test Case # Version Description PON # Date/Time Sent To Verizon Pass/ Fail Issues

  1. AA Change Hunting from C to S, Add additional DL TESTPON01 3/22/99 6:30 p.m. Fail There should not be an ALI on second DL form because LACT is N AB 4/01/99 10:00 a.m. Fail REQTYP missing AC Pass LSC sent
  2. AA Change Hunting from P to M, change to Non-Pub TESTPON02 3/22/99 6:30 p.m. Pass LSC sent
  3. AA Remove Foreign Listing TESTPON03 3/28/99 1:00 p.m. Fail Failed in EDI for incorrect EDI sequence. Please review LSR to insure EDI segments in correct order. AB 3/30 11:50 a.m. Fail Initiator ID and Initiator tel no not sent.

View entire OSS Interface Change Management Process Document

|||||||||||||||||