Документ взят из кэша поисковой машины. Адрес оригинального документа : http://www.eso.org/projects/dfs/team/integration-proc.html
Дата изменения: Fri Jul 29 16:47:54 2005
Дата индексирования: Sun Apr 13 22:49:01 2008
Кодировка:

DFS Commissioning: integration procedure

DFS Department
DFS integration testing process

HOME

INDEX

HELP

NEWS

The DFS integration testing procedure involves each DFS Department role : the DFS Department Head, several DFS project managers and developers, the SEG testers and the DFS Department QA manager.
This procedure implies 5 constraints:

Apply the expected schedule for deliveries to SEG
Draw up a standard Release Note for each delivery to SEG
Hold integration kick-off meeting(s) for every new DFS component release
Describe integration test results via a standard Test Report
Record new bugs and action items found during integration tests
Hold integration ending meeting(s) for every new DFS component release

Following procedure is applicable to every DFS component.
For instrument pipelines (UVES, ISAAC, ...), a specific arrangement applies.

First, some definitions ...

The Package Deadline corresponds to the date by which a new DFS package has to be delivered OUTSIDE the DFS Department.

a DFS package may contain a complete new DFS release, only some DFS components new releases or even only one DFS component new release, depending on the delivery goal.

A DFS package may:

directly be used for operations (case of a P2PP or a PHRS delivery to external users, for instance)
or, first installed on-site for commissioning tests prior to operations (case of a DFS commissioning on the mountain or an instrument commissioning).

For DFS releases corresponding to a DFS commissioning, the Package Deadline usually corresponds to the day before the departure to the mountain.

An official release is the one corresponding to the DFS package delivered outside the DFS Department.

Before being able to get an official release, i.e. a release successfully tested by the DFS group through Unit tests and through Integration tests, several beta-releases have usually to be issued. A beta-release is then a specific temporary version of a DFS component delivered to SEG for integration test purposes only.

A beta-release should NOT be used operationally, unless there is a formal agreement from the DFS Department Head, and appropriate warnings and recommendations to end-users: in principle, only official releases, i.e. releases successfully tested by the DFS Department can be used operationally.

A release classification is defined according to the expected content of the official release :

a major release usually contains a large set of change requests and bug fixes and/or major changes such as port to another coding language, support a new OS, ...
a minor release contains less important modifications like a data or database format change, some new basic functionalities, ...
a patch release is basically a release addressing a limited number of fixes for critical/blocking bugs found during operations.

The release type major or minor is defined for each DFS component through SCCB meetings and/or DFS commissioning kick-off meetings.

A release should NEVER contain changes or fixes which were not officially decided via SCCB meetings or DFS commissioning kick-off meetings or, first formally agreed by the DFS Department Head.

Go to top of page

Apply the expected schedule for deliveries to SEG

For any new DFS component delivery to the mountain or external users (astronomical community) corresponding to a major or a minor release, at least 1 beta-release has to be delivered to SEG for integration/validation tests.

For patches, a beta-release is usually not necessary, since it has to be operational very quickly and the number of changes comparing with the previous version should be limited. Anyway, beta-releases can also be delivered to SEG for patches.

For any DFS component, the first beta-release must be delivered to SEG for integration/validation tests not later than:

4 weeks before the Package Deadline, for any major DFS component release.
4 weeks before the Package Deadline, for a minor DFS component release delivered for a DFS or an instrument commissioning event.
2 weeks before the Package Deadline, for a minor DFS component release NOT related to a commissioning event.
if a beta-release has been decided for the patch, 2 weeks before the Package Deadline.

The first beta-release must contain ALL expected features and functionalities decided through SCCB meetings and/or DFS commissioning kick-off meetings, not more, not less.

For any DFS component, the last beta-release must be delivered to SEG for integration/validation tests not later than:

2 weeks before the Package Deadline, for any major DFS component release.
2 weeks before the Package Deadline, for a minor DFS component release delivered for a DFS or an instrument commissioning event.
1 week before the Package Deadline, for a minor DFS component release NOT related to a commissioning event.
if a beta-release has been decided for the patch, 1 week before the Package Deadline.

Beta-releases must contain same new items as the first beta, plus ONLY fixes of bugs detected during integration testing.

The official release itself, i.e. the one to be installed on the mountain or delivered to external users, must be delivered to SEG not later than:

1 week before the Package Deadline, for any major DFS component release.
1 week before the Package Deadline, for a minor DFS component release delivered for a DFS or an instrument commissioning event.
3 days before the Package Deadline, for a minor DFS component release NOT related to a commissioning event.
for a patch, 3 days before the Package Deadline.

These delays include the basic non-regression tests to be performed on an official release and DFS package preparation durations.

An official DFS component release must NOT contain any new feature (or change request) compared with the last beta-release, nor fixes of minor bugs possibly originated in previous beta-releases: in other words, it can contain ONLY fixes of remaining major bugs possibly originated on the last beta-release. This rule should be applied as much as possible, in order to avoid introducing regressions or bad side effects with last minute changes. Exception to that rule can be accepted, but only after a preliminary agreement between developers and SEG testers.

Go to top of page

Draw up a standard Release Note for each delivery to SEG

A Release Note has to be written by the Project Manager for every new DFS component delivery to SEG (i.e. for each beta-release AND for the latest official delivery), even if it is only related to a new library version and maybe not to an application change.

The Release Note has to be archived via the webcheck tool, at least 1 week before the corresponding first beta-release is delivered to SEG. Release notes will then accessible directly from the DFS group Home Page.

When publishing a new Release Note, please, send also an email to inform SEG.

For other following beta-releases and for the official one, the Release Note has to be send as soon as possible before the delivery to SEG, in order for SEG to:

plan, prepare and perform generation and installation of the new release on the test environment.
review and update the existing testcases, if necessary.
provide/retrieve any new or specific test data.

The Release Note will be a text file containing following items :

the DFS component name: use the standard DFS components abbreviations only.
the corresponding release number. The numbering rules are:
- xx for a major official release.
- xx_yy for an minor official release.
- xx_yy_zz for a patch official release.
A beta-release number is added _beta i for beta-releases: e.g. P2PP-2_2_beta5 designates the fifth beta version of the 2_2 release.
Name of the DFS Department member who delivers the new release to SEG, and corresponding date.
the release type: commissioning, patch, other (see above).
the main reason of the delivery : a patch to fix as quickly as possible a critical bug, release to solve a set of bugs and change requests, new DFS component able to run with a new operating system version, a new compiler version, a new database structure, a specific release to support an instrument dry-run, ...
the final expected DEADLINE for delivery outside DFS Department (=Package deadline): essential especially when a critical/blocking bug is to be fixed and solved as quickly as possible. According to it, SEG will set the highest priority to re-run integration tests and prepare the delivery package.
description of the required configuration for integration tests (OS, VLT software, libraries, ..., versions to be used for integration, specific configuration features to apply (e.g. a specific DB server, ...).
the list and location of any specific test data files to be used during integration (new instrument package, specific FITS files, ...).
a list of features already checked during Unit Tests. Any result from tools like Purify and Quantify are strongly recommended.
a list of features to be carefully checked during integration tests: specific checks, any advice/recommendation from the developer to the integration tester, ...
the complete list of action items (extracted from associated To-Do list) and/or DFS Problem Reports addressed by this (beta)release. Each release note (for beta and for official releases) contains only new items compared with the previous official release note, classified by previous beta-releases. For instance, release note for P2PP-2_1_2_beta4 contains items specific to the last beta4, then items specific to P2PP-2_1_2_beta3, then to P2PP-2_1_2_beta2, etc.
the complete list of action items / Problem reports which are NOT solved yet by the current (beta-)relelase; it maybe be a web link to such list. Furthermore, it may very useful to indicate here important problems which are not solved yet, while they should.
the list and issue of any updated documentation : User's manual, software specification, design document, ICD, ...
any additional comments which may be useful for generation/installation or testing purposes.

The following link gives access to the standard template for DFS Release Notes.
A Release Note is to be described via a simple text file named:
if it's an official release,

"DFSComponentAbb-xx.txt" for a major release,
"DFSComponentAbb-xx_yy.txt" for an minor release,
"DFSComponentAbb-xx_yy_zz.txt" for a patch release,

if it's a beta-release,

"DFSComponentAbb-xx_betai.txt" for a major release,
"DFSComponentAbb-xx_yy_betai.txt" for an minor release,
"DFSComponentAbb-xx_yy_zz_betai.txt" for a patch release,

where DFSComponentAbb designates the standard DFS components abbreviations.

Go to top of page

Hold an integration kick-off meeting for every new DFS component release

As soon as the Release Note is available, an integration kick-off meeting has to be held at least between the Project Engineer and the Integration tester in order to:

review all change requests and bugs to be tested and check that the problem description is understandable.
list all existing and new integration testcases to be run.
check that test data to be used will be available in due time.
and then, check that the release note content is consistent with the template defined above.

Minutes of this integration kick-off meeting should be written using the corresponding standard template.

The DFS Department Head, Project manager and DFS Department QA manager should be informed of this meeting in due time to be able to participate if needed. They should also receive the corresponding minutes.

Go to top of page

Describe integration test results via a standard Test Report

Once integration tests are performed, a Test Report has to be established according to the standard template.

It has to be sent first to the DFS Quality Assurance manager, who will verify that the corresponding template is applied, and then put it as available from the DFS Department Home Page.

Go to top of page

Record new bugs and action items found during integrations tests

If SEG testers find new bugs or problems during integration tests, they have to record them under Action Remedy. They can also use DFS To-Do lists.

Each concerned Project Manager and/or engineer has to review the Test Report and, if required, open other specific action items in DFS To-Do lists (for example, according to Test Report results, ask for more tests, get other test data, run another check, ...).

To-Do lists and SPRs have to be regularly updated. To-Do lists are reviewed during every corresponding DFS bi-monthly project meeting held with the DFS Department Head. SPRs are reviewed during SCCB meetings.

Go to top of page

Hold an integration ending meeting for every new DFS component release

If a SEG tester finds critical bugs or problems during integration tests, a specific meeting has to take place with the concerned DFS Project manager and developer(s), SEG tester, DFS Department Head, and possibly the ESO Project scientists or representative end-users, to decide what to do: fix the bug if possible, define a workaround, postpone the delivery, ....

This type of meeting can be held on a case by case basis, i.e. even if there are no critical issues to review, simply to discuss integration test results, and plan next integration or delivery steps.

DFS Department DFS integration testing process