PROCEDURES TO INTRODUCE SUPERCOMPUTERS

19 April 1990

The 13th Meeting of the
Committee for Drawing Up and
Promoting the Action Program

Decision of the Committee for Drawing Up and Promoting the Action Program

The Government of Japan, has decided upon "Procedures to Introduce Supercomputers," at the 16 July 1987 meeting of the Committee for Drawing Up and Promoting the Action Program, and has endeavored to implement the Procedures accordingly, as from 1 August 1987. In consideration of, among others, the status of the implementation of the Procedures thus far, the Government decided to revise the "Procedures to Introduce Supercomputers," as outlined herein, in order to ensure further transparent, open, and non-discriminatory competitive procedures for the introduction of supercomputers and to make it easier for a procuring entity to procure the supercomputer that best enables it to perform its mission.



Procedures to Introduce Supercomputers

It is the policy of the Government to provide transparent, open and non-discriminatory competitive procedures for the introduction of supercomputers and to ensure that a procuring entity procures the supercomputer that best enables it to perform its mission. The following Procedures are adopted in order to implement that policy fully and effectively. The Procedures specified herein are intended to address the changes in the procurement environment identified to date.

A competitive procedure will mean there is no tendency, whether intentional or unintentional, for any company domestic or foreign to be favored, hindered, or blocked in its ability to supply a machine which meets the information processing needs of prospective users. It is not the policy of the Government to procure supercomputers based on bids that contravene the prohibition in the Anti-Monopoly Act against unjust low-priced sales. These Procedures go into effect as from May 1, 1990.

These Procedures will be implemented while ensuring consistency with the requirements of the GATT Government Procurement Code, as amended (hereinafter referred to as "the Code").

I. Scope and coverage

  1. These Procedures govern the introduction (purchase, rental and lease) of supercomputers by the entities subject to the Code.

  2. Each ministry/agency concerned will give guidance to quasi-governmental agencies within its jurisdiction which are planning to introduce supercomputers to follow procedures in line with the purport of these Procedures.

  3. These Procedures will apply to the introduction of supercomputers with theoretical peak ratings of 300 million floating point operations per second (MFLOPS) and over. This coverage will be reviewed as necessary.

II. Procedures

Procedures stipulated by the Code and the Action program for Improved Market Access will be followed for all supercomputer procurements. Under the principles of free competition, all procedures will be conducted in a manner which ensures national treatment and non-discrimination. From this viewpoint, an entity which hereafter plans to introduce a supercomputer (hereinafter referred to as "the entity") will follow the Procedures outlined below with a view to supplementing the Code procedures and precisely implementing its principles.

  1. Procedure for market research

    1.1. Invitation to submit material

    (1) When an entity has determined that it has needs which may be satisfied by the introduction of a supercomputer, it will draw up its actual minimum needs, including parameters of forecasted workloads based on anticipated use. These parameters will include minimally acceptable benchmark results which demonstrate the operational performance of the supercomputer required by the entity. The parameters may also include such requirements as main and secondary memory size. Where a specific rate of operational performance is required within a fixed period of time, that rate may be included in actual needs; however, it will be cut in such a manner as not to discourage potential suppliers.

    (2) The entity will conduct market research for the purpose of ensuring the development of actual minimum needs. The entity will collect information on actual market prices for supercomputer systems in similar working environments in the private sector, unless impossible. This research will be conducted in a transparent manner and will serve as the basis for establishing the estimated contract price and requesting a budget sufficient for introduction of the system.

    (3) In requesting information from a supplier, the entity will not provide or deny information in such a manner as would give any supplier a discriminatory advantage. The entity will not provide advance knowledge concerning a future plan to introduce a supercomputer to any prospective supplier, except according to procedures described in (4) below. Suppliers who are directly involved in the development of the actual minimum needs shall be barred from participating in the tendering process.

    (4) The entity will make an announcement in the Kanpo (hereinafter referred to as "the announcement") of its plan to introduce a supercomputer in accordance with the actual minimum needs developed in (1) above, and issue an invitation to suppliers for submission of general information material and comments on the basic need requirements (including specifications and other technical information). Simultaneously, supplementing this, the entity will send an invitation letter (hereinafter referred to as "the letter") to those domestic and foreign suppliers of which it has knowledge, including those that have expressed an interest in supplying supercomputers. Equal treatment shall be accorded to those suppliers responding to the announcement and those responding to the invitation letter.

    (5) The announcement and the invitation letter mentioned above will be given or sent at least 40 calendar days prior to the last day of the period set for receiving submission of the said material and comments from suppliers.

    (6) The announcement and the invitation letter will contain the following:

    (a) the plan to introduce a supercomputer and actual minimum needs thereof;

    (b) the deadline for submission of material and comments;

    (c) notice that the introduction documentation will be sent, upon request, to suppliers responding to the announcement or the invitation letter as well as the place where and the period when suppliers can obtain the introduction documentation; and

    (d) notice of an explanation session for the introduction of a supercomputer, if held.

    1.2 Introduction documentation

    (1) Suppliers responding to the announcement or the invitation letter will be, upon request, provided with the relevant introduction documentation by the entity.

    (2) The introduction documentation will contain, inter alia, the following:

    (a) the place where suppliers' material must be submitted (contact points);

    (b) the place where suppliers' inquiries will be received or additional information can be obtained;

    (c) the deadline for submission of material;

    (d) detailed requirements of the supercomputer planned to be introduced (required performance, type of operation and others);

    (e) benchmark materials covering representative forecaster workloads of the entity;

    (f) the date and place of the explanation session for the introduction of a supercomputer, if held; and

    (g) the objective criteria for evaluating each mandatory requirement and for evaluating the technical excellence of the bids.

    1.3. Explanation session for the introduction of a supercomputer

    The entity will hold, as necessary, an explanation session with regard to the introduction documentation. In case the date and place is not contained in the introduction documentation, it will send an invitation to all suppliers responding to the announcement or the invitation letter, while allowing sufficient time for suppliers to consider the information.

    1.4. Inquiries

    (1) The entity will promptly respond to inquiries made by suppliers with respect to the content of the announcement, invitation letter or introduction documentation.

    (2) When the entity makes any amendment to or has any additional information concerning the introduction documentation, it will simultaneously provide the amendment or additional information to all interested suppliers responding to the announcement or the invitation letter and allow sufficient time for suppliers to submit additional material so that they can consider and respond to such amendment or information.

    (3) The entity may ask questions or make inquires to suppliers with regard to submitted material, but will not do so in a manner that provides a discriminatory advantage to any supplier. The entity may also, as necessary, conduct research including performance and function verification with respect to submitted material.

    (4) The entity will not disclose or divulge material or information provided by a supplier without its consent to any third party, including other supplies.

    1.5. Benchmark testing

    (1) Benchmark testing definition: For the purposes of these Procedures, benchmark testing is defined as "The measurement of achieved performance of a supercomputer on a representative forecasted workload (set of codes), designated by the customer, and as measured by the elapsed time on the wall clock, as utilized in the United States."

    (2) Benchmark testing will be conducted in every procurement in accordance with the following provisions:

    (a) The procuring entity will specify the operational performance of the supercomputer that will be required and that is commensurate with the entity's actual minimum needs, as specified in Section II.1.1(1), "Invitation to submit material."

    (b) The entity will supply all necessary documentation for benchmarks as set forth in Section II.1.2(e), "Introduction documentation."

    (c) The selection of benchmarks will be representative of the anticipated workload of the entity, as described in Section II.2.1(2), "Preparation of specifications."

    (d) The results of benchmark tests conducted in accordance with Section II. 3.5(2), "Technical examination," will be submitted with the bid.

    (e) The entity may conduct benchmark testing subsequent to awarding a bid to verify previously submitted benchmark results.

  2. Specifications

    2.1 Preparation of specifications

    (1) The entity will prepare technical specifications based on the actual minimum needs it identified during the market research phase. If the procurement will replace or interconnect with an existing system, these specifications will be prepared in such a way that suppliers can compete effectively against the incumbent supplier. Features not essential to the assigned operational tasks will not be required.

    (2) Benchmarking tests based on a representative forecasted workload of a procuring entity will be specified in final form in the solicitations so that functional performance is used as the basis of the supercomputer's technical evaluation. The entity will provide all the documentation necessary for the supplier to prepare and execute the specified benchmark tests. Operating system and other software capabilities essential to the system's performance will be listed in the specifications.

    (3) Overall systems performance will be emphasized over equipment specifications so that bidders can concentrate on satisfying actual user needs. Operation Performance, as described in Section II.1.1(1), may be used in the specifications to describe the entity's general concept of the capability of the supercomputer to be procured.

    (4) Specifications will contain all the information necessary for the potential suppliers to understand the entity's actual minimum needs. Suppliers who are directly involved in the actual development of specifications will be barred from participating in the tendering process.

    2.2 Explanation of specifications

    (1) The entity will send an invitation to all the suppliers responding to the announcement or the invitation letter and explain to them its specifications developed under Section II. 2.1. If the entity plans to use single tendering procedures based on compatibility requirements, Code justifications will be included.

    (2) If the entity requires interoperability with existing hardware and/or software, it will accomplish this objective in a manner that does not entail involuntary or unauthorized disclosure of proprietary operating systems, interfaces or protocols.

    2.3 Inquiries and proposals

    The entity will give suppliers at least 50 calendar days after the above explanation thereby providing opportunities for the entity and suppliers to make inquiries, and for suppliers to submit proposals and revise their proposals concerning the specifications.

  3. Tendering procedures

    In the procurement of a supercomputer the entity will follow the procedures set forth in the following sections.

    3.1. Basic procedures

    (1) Purchase

    Procedures stipulated by the Code and the Action Program for Improved Market Access will be followed.

    (2) Rental-Lease

    Procedures stipulated by the Code covering rental-lease will be followed.

    3.2. Bid deadlines and provision of information

    (1) The entity will, when procuring a supercomputer by competitive tendering procedures, ensure that the period for receipt of tenders be at least 40 calendar days from the date of issuance of the invitation to tender.

    (2) The entity will, when procuring a supercomputer by single tendering procedures based on the compatibility requirements, provide in the Kanpo information concerning the procurement plan together with the compatibility requirements at least 40 calendar days prior to the conclusion of the contract and after the period mentioned in Section II. 2.3. The entity will promptly provide related information to suppliers who make inquiries to it based on the information in the Kanpo.

    3.3. Finalization of specifications

    After any inquires and/or proposals regarding the solicitations have been made, and all parties have been informed of any changes, as presented in Sections II.1.4 and 2.3 above, the entity will finalize its specifications and provide them to the interested bidders.

    3.4. Bidding procedures

    (1) Bids will be evaluated based on overall greatest value to the entity which is determined by considering technical and functional performance factors as well as price. The specific criteria for evaluation of a procurement are specified in Section II.3.7.

    (2) The procuring entity will establish the estimated contract price for the introduction of a supercomputer based on the prices for similar supercomputer systems in similar working environments in the private sector, unless impossible.

    (3) When one or more bids are offered at a price equal to or less than the estimated contract price established by the procuring entity and meet the minimum functional performance thresholds established by the procuring entity, additional rounds of bidding will be expressly prohibited. Thresholds for functional performance are established in Section II.2.1. The established contract price, which will reflect an adequate budget, will be decided in accordance with (2) above.

    (4) The procuring entity will not require additional bidding rounds when only one supplier is participating, once that supplier meets functional performance requirements established in Section II.2.1 and has offered a price that is equal to or less than the estimated contract price that has been established in accordance with (2) above for the introduction of a supercomputer.

    3.5. Explanation session

    The entity will hold, as necessary, an explanation session with respect to the final specifications and requirements. In such a case, identical information will be provided to all suppliers wishing to participate in the tendering.

    3.6 Technical Examination

    (1) Benchmarking: The entity will conduct the previously determined and specified benchmark testing of a representative workload. For purposes of technical evaluation, entities will use solely those benchmarks described in the specifications. The selection of benchmarks will be based on the anticipated workload of the entity as stated in Section II.1.1. The entity will give each bidder adequate advanced notice before benchmark tests are run on its system. The only criteria to be used in performance testing will be those specified in Section II.2. The tests will not be run under such conditions and terms as would give any supplier a discriminatory advantage.

    (2) Benchmark testing will be conducted on an existing system in every procurement exception procurements that meet the following conditions:

    (a) a supplier offers the first of a new model machine that the supplier may not yet be able to benchmark;

    (b) if a supplier meets the requirements of (a) above, the other suppliers in that procurement will be allowed to offer a future model that the suppliers may not yet be able to benchmark;

    (c) the supplier with the winning bid must deliver the machine by the announced delivery date or the entire procurement will be subject to rebidding; and

    (d) the winning system must, before delivery, satisfy the benchmark tests with rest equal to or better than the forecasted results and the specifications.

    (3) The entity will show the results of the benchmark testing and the verification upon request from any other bidder after the system is delivered.

    (4) The bidders may file 2 complaints with respect to the benchmark testing and its results with the Procurement Review Board.

    3.7. Award Criteria

    (1) The overall evaluation of bids will be conducted in a manner that ensures equal treatment of all bidders and full transparency. In the overall evaluation of bids, the procuring entry will consider technical excellence, with overall system performance being of fundamental importance, and cost. To be eligible for contract award, the proposals must be within the estimated contract price determined in accordance with Section II.3.4 (2) and meet the mandatory requirements set forth in the specifications in Sections II. 2 and 3.3.

    (2) The entry will specify the mandatory and other requirements for the supercomputer system which it needs to procure. The criteria for evaluating each mandatory requirement will be on a pass/fail basis and will be specified in the introduction documentation in Section II. 1.2 (2) (g) and final specifications in Section II. 3.3. Only those proposals which pass all mandatory requirements will be further considered.

    (3) The procuring entity will also specify the objective criteria for evaluating the technical excellence of the bids in the introduction documentation in Section II. 1.2 (2) (g) and in the final specifications in Section II. 3.3. The objective criteria will reflect the scoring system for each technical factor, including those mandatory requirements which can be exceeded for extra credit. Also, when extra credit is given under a particular technical factor for providing a non-mandatory requirement, this will be included as part of the objective criteria. No credit will be given for providing features not included in the final specifications. Bidders may propose changes to any part of the evaluation criteria during the inquiry stage. The overall evaluation of the technical proposal will be measured by aggregating the score for each technical factor.

    (4) Software availability will be taken into consideration.

    (5) The entity will determine the award of the contract by analysing the technical as well as the cost factors on the basis of the most favourable proposal to the procurer.

    3.8. Final contract price

    The final contract price will be decided in accordance with the award criteria set forth in section II. 3.7.

    3.9. Unfair bids

    (1) It is not the policy of the Government to procure supercomputers based on bids that contravene the Anti-Monopoly Act, including the prohibition against unjust low-priced sales.

    (2) Where a bid is submitted that, because of its price or other terms, unlawfully impedes fair competition, the bid will be deemed held in its entirety and the procuring entity will not consider that bid in awarding the supercomputer contract.

    (3) Any bidder that submits a bid referred to in (2) above will, as a matter of principle, be deemed ineligible to resubmit a bid in that supercomputer procurement; and the name of such bidder will be announced publicly.

    (4) If, after a contract is awarded, 2 complaint is filed pursuant to Section III and the FTC or the Court determines that the bid upon which the award was based unlawfully impeded fair competition, the procuring entity will take the most appropriate action as set forth in Section II. III4.4.

  4. Post-awarded information and debriefing of the outcome of the procurement

    Consistent with the requirements of Article VI of the Code, the entity will publish information on the contract award and, upon request from an unsuccessful supplier who has submitted material, promptly provide such supplier with pertinent information concerning the reasons for not being selected, including the name of the selected system as well as the information on its relative advantages. The information may, however, exclude such as would prejudice the legitimate commercial interests of particular suppliers or might prejudice fair competition among suppliers.

III. Complaint Mechanism for Supercomputer Procurements

  1. Overview

    In order to provide for fair and open competition and to achieve consistency with the provisions of these Procedures in procurements of supercomputers, the following complaint process will be effective 30 days after these Procedures go into effect.

  2. Procurement Review Board

    2.1. The Government will establish a Procurement Review Board (Board), as an independent reviewing organ, to review complaints by potential suppliers of any aspect of 2 procurement of a supercomputer subject to these Procedures. The Board will have no substantial interest in the outcome of a supercomputer procurement subject to its review.

    2.2. The Board will receive complaints in writing, conduct investigations of the facts and make recommendations with respect to any aspect of a procurement of a supercomputer by an entity.

    2.3. The Board will be comprised of persons who have knowledge and experience related to public sector procurements. No member of the Board will participate in the review of a complaint in which that member has a conflict of interest.

  3. Procurement Review Process

    3.1. A potential supplier may file a complaint with the Board when it believes the procurement has been carried out in a manner inconsistent with the intent or any provision of these Procedures. It may also file a complaint based upon the allegation that the contract was awarded to a supplier that had submitted a bid in violation of the Anti-Monopoly Act. Potential suppliers are encouraged to seek resolution initially with the entity of any alleged inconsistency with these Procedures.

    3.2. Timing of complaints

    (1) A complaint may be filed at any time during the procurement process, but no later than 10 days after the basis of the complaint is known or reasonably should have been known. The potential supplier will submit a copy of the complaint to the entity within one day of filing it with the Board.

    (2) The Board may consider a complaint, even though not timely filed, if it finds that good cause is shown or that a complaint raises issues significant to the purpose of these procedures.

    3.3. The Board will review a complaint within seven days of its filing and may, in writing and with good reasons given, dismiss any complaint found to be:

    (1) not submitted in a timely manner,

    (2) not subject to these Procedures,

    (3) frivolous or trivial on its face,

    (4) not submitted by a potential supplier, or

    (5) otherwise inappropriate for review by the Board.

    3.4. Where the Board determines that a complaint has been filed properly, it will notify in writing all potential suppliers within one working day of the complaint.

    3.5. Suspension of award or procurement process

    (1) Within 10 days of the filing of 2 pre-award complaint, the Board will issue written requests for suspension of the procurement process pending resolution of the complaint.

    (2) Within 10 days of the filing of 2 post-award complaint, the Board will request in writing suspension of performance of the contract pending resolution of the complaint.

    (3) The entity will, as a matter of principle, suspend the procurement process or performance of the contract immediately after it receives the Board's request, unless the head of the entity determines that urgent and compelling circumstances do not allow the entity to follow the request, in which case he will immediately inform the Board of his determination.

    3.6. Investigation

    (1) The Board will conduct an investigation of the complaint, which may include the filing of briefs, pleadings and other documentation by the complainant and entry.

    (2) The Board may, on the request of the complainant or entity or on the Board's own initiative, hold a hearing on the merits of a complaint.

    3.7. Entity Report

    (1) Within 25 days after the day on which a copy of the complaint was sent to the entity, it will file with the Board a complete written report on the complaint, containing the following:

    (a) The solicitation, including the specifications or portions thereof relevant to the complaint;

    (b) all other documents relevant to the complaint;

    (c) a statement that sets out all findings, actions and recommendations of the entity and responds fully to all allegations of the complaint; and

    (d) any additional evidence or information that may be necessary in order to resolve the complaint.

    (2) The Board will, forthwith after receiving the report referred to in (1) above, send a copy of the relevant material to the complainant and give the complainant an opportunity, within seven days, after it receives the relevant material, to file with the Board comments or request that the case be decided on the existing record. The Board will forthwith after receiving the comments, send a copy to the entity.

    3.8. Participants: The entity and potential suppliers whose direct economic interest would be affected by the award of, or the failure to award, a contract may participate in a complaint proceeding.

  4. Findings and recommendations

    4.1. The Board will make a report of its findings and recommendations within 90 days after the day on which the complaint is filed. Its findings, in which the Board will grant or deny the complaint in whole or in part, will specify whether the procurement process or award was inconsistent with the intent or specific provisions of these Procedures.

    4.2. (1) Where the Board finds that there is a significant likelihood that a contract was awarded to a supplier that had submitted 2 bid that contravened the prohibition in Anti-Monopoly Act against unjust low-priced sales, it will report the case to, and request that, the Fair Trade Commission (FTC) determine whether or not the Anti-Monopoly Act has been violated and take appropriate measures.

    (2) Pending the notification by the FTC of its final findings, the Board will request the entity to suspend performance of the contract. With this request, the entity, as a matter of principle, will suspend performance of the contract. Upon receipt of the FTC's notification, the Board will complete its review of the complaint and where the FTC found a violation of the Anti-Monopoly Act, the Board will recommend that the entity adopt a remedy set forth in Section III. 4.4.

    4.3. In making its findings and recommendations, the Board will consider all the circumstances surrounding the procurement process or award, including the seriousness of the deficiency in the procurement process, the degree of prejudice to ail potential suppliers or to the integrity and effectiveness of these Procedures, the good faith of the participants and the extent of performance of the contract to which the procurement relates.

    4.4. Where the Board finds that the intent or any provision of these Procedures has not been realized, it may recommend an appropriate remedy, including one or more of the following:

    (1) that a new tender package be issued,

    (2) that new bids for the contract be sought,

    (3) that the bids be re-evaluated,

    (4) that the contract be awarded to another supplier, or

    (5) that the contract be terminated.

    4.5. The Board will send its findings in writing with its recommendations to the complainant, the entity, and any other potential suppliers, within one working day after issuance. The Ministry of Foreign Affairs will respond to external inquires concerning the findings.

    4.6. Where the entity does not follow the Board's recommendation, the entity will send a copy of its decision, and the rationale for it, to the Board within one working day after issuance. The Ministry of Foreign Affairs will respond to external inquiries concerning the decision.

    4.7. Wherever the Board discovers evidence of misconduct, or behavior contrary to law or regulation, such evidence will be referred to law enforcement authorities for appropriate action.

  5. Express Option

    5.1. Where the complainant or entity requests in writing an expeditious handling of a complaint, the Board will consider the feasibility or using the procedure set out in this section, referred to herein as the "express option."

    5.2. The Board will determine whether to apply the express option within two working days after receiving a request therefore, and will notify the complainant and entity as to whether the express option is to be applied.

    5.3. Where the express option is applies, the time limits and procedures will be as follows:

    (1) The entity will, within 10 days after the day on which it is notified by the Board that the express option is to be applied, file with the Board a complete report on the complaint, as specified in Section III. 3.7. The Board will, forthwith after receiving the report, send a copy of the relevant material to the complainant. The Board will give the complainant five days to file with the Board comments on such material or request that the case be decided on the existing record. The Board will, forthwith after receiving the comments, send a copy to the entity.

    (2) To Board will issue its findings and recommendations on the complaint in writing 45 days after the day on which the complaint is filed.



Index | Back | Next