Step 1: Initiate

· What types of projects and services does the QAT Team support?

o The QAT Team supports various IT projects and services, including software applications, websites, databases, and infrastructure projects. They are involved in both new development and ongoing enhancement activities.

· What is the Texas Project Delivery Framework (TPDF)?

o Good, consistent project management helps organizations successfully deliver projects on budget and on schedule. To make project management easier, the Texas Department of Information Resources (DIR) has collaborated with the QAT and many statewide agencies to create the Texas Project Delivery Framework Templates, based on PMBOK standards, as the required templates for major information resources projects.   

· What is a Major Information Resources Project?

o A “major information resources project” is any information resources technology project identified in a state agency’s Biennial Operation Plan (BOP) that:

§ Is specifically identified as such by the Legislature in the General Appropriations Act; or

§ Has development costs of $5.0 million or greater and:

§ Requires one or more years to reach operational status; or

§ Involves more than one state agency: or

§ Substantially alters work methods of state agency personnel or the delivery of services to clients.

The term also includes any information resources technology project of a state agency designated for additional monitoring under Texas Government Code, Section 2261.258(a)(1), if the development costs for the project exceed $5 million (Texas Government Code, Section 2054.003(10)).

· If our project has a total project cost of $5 million, but only $4 million of that is to pay for software development, and the rest is for staff augmentation services and hardware costs, is the project still classified as a MIRP?

o The development costs for a project include all project costs associated with the project’s implementation. While a development project often has the largest costs associated with software purchase and customization development, all costs associated with the successful implementation of the project are to be used to determine the project’s MIRP eligibility. Costs such as staffing, hardware, software licenses, contingency, and professional services are all considered development costs and are calculated toward the total project cost. The project described in this scenario would be classified as a MIRP.

· Does QAT provide training on how to complete Framework Documents? I’d like to have my team trained.

o Yes! Webinars covering the Framework Templates are available 24/7 on the QAT website. You can also request individualized training via email at projectdelivery@dir.texas.gov.

· My project is for an institution of higher education (IHE). Is it exempt from completing Framework documentation?

o Subsequent to September 1, 2021, QAT oversight may apply to any information resources technology project of an institute of higher education assigned additional monitoring pursuant to Texas Government Code, Section 2261.258(a)(1), if the development costs for the project exceed $5.0 million. Details on Additional Oversight requirements are available in the QAT Policy and Procedures Manual and on the DIR website.

· What if my MIRP originates outside of the legislative planning cycle?  Is there anything I need to do in order to meet requirements?

o If your project is not included in your agency's current Biennial Operating Plan (BOP), you must submit a Biennial Operating Plan (BOP) Amendment to the Legislative Budget Board (LBB). Once approved by the LBB, you must submit a Business Case Template, Business Case Workbook, and Statewide Impact Analysis to the QAT.

· We made a request in our LAR for this project, but the scope and approach has changed since that submission. Are we required to do anything to make this change and notify QAT?

o If project scope/approach changes are needed after LAR approval by the Legislature, the agency must follow its project change process to document the change and submit a revised Business Case Template to QAT via the Statewide Project Automated Reporting (SPAR) system and by email at QAT@dir.texas.gov. Also, describe the change in the first Monitoring Report Section 6, Project Changes.

· For our agency, it is sometimes more efficient to combine multiple enhancements over 80 hours into one big project, developed by the maintenance team. If that causes the project to exceed MIRP thresholds, does that mean it must be reported as a MIRP?

o Any enhancement effort over 80 hours that is also over $5 million in total project costs must be reported as a MIRP.

· If our agency defines all unresolved defects and new enhancements to the original scope as backlog, which is considered maintenance, not development, does that work qualify as a MIRP?

o Any defects not resolved by the close of the project should be considered a vendor nonperformance issue and should be addressed accordingly through the contract. These already identified defects are not considered “maintenance,” as they are part of the development scope and should not be accepted by the agency if not satisfactory. Any work defined as “backlog” or “maintenance” by an agency does not relieve the agency from requirements for IT Project and MIRP reporting as detailed in TGC, Chapter 2054; GAA, Article IX; and TAC, Chapter 216.

Bottom line, if you are making changes to an application that total greater than 80 hours of work, that is development regardless of the team it is assigned to, and is subject to TGC, Chapter 2054; GAA, Article IX; and TAC, Chapter 216, requirements for IT Project and MIRP reporting.

· We are on phase 3 of this project, which includes as scope resolving defects from phase 2 and then the data migration. Is this a MIRP?

o A legacy data migration effort over 80 hours that is also over $5M in total project costs must be reported as a MIRP. 

· We’d like to follow the QAT guidance to split up large projects into phases rather than report as one project. Would taking this approach mean this is still considered a MIRP?

o If the large project extends over multiple biennia, the recommendation is to split the project up into phases for each biennium.  If each biennial phase of the project’s total costs are $5 Million or higher, each phase would be considered a separate MIRP. If the agency is appropriated a large amount of money to be spent in a single biennium for a project, splitting up that project into smaller projects moving toward the same ultimate goal is not part of QAT guidance.  Ultimately, the information that is in the agency’s BOP when requesting the appropriation is the basis for MIRP determination. 

· We are considering a Software as a Service (SaaS) solution for this project. Does that mean this is not a MIRP?

o This depends greatly on whether the SaaS solution requires customization in order to implement the scope.  If the SaaS is undergoing minimal configuration as part of implementation, that MAY be considered for MIRP determination.  However, if the SaaS is undergoing over 80 hours of customization, that would indicate a level of development that is consistent with a MIRP.  The QAT always recommends having a discussion to determine the specifics of each project’s scope, costs, and schedule can vary widely.   

· Our agency doesn’t refer to IT efforts as “projects.” Therefore, there isn’t anything in our portfolio that would meet the definition of a Major Information Resources Project. Is there anything we need to do to report this to QAT?

o A project is defined in statute as any initiative that provides information resources technologies and creates products, services, or results within or among elements of a state agency and is characterized by well-defined parameters, specific objectives, common benefits, planned activities, a scheduled completion date, and an established budget with a specified source of funding (Texas Government Code, Section 2054.003, and Texas Administrative Code, Chapter 216). Texas agencies are required to define projects in alignment with statutory requirements, any agency-specific terminology does not eliminate the requirement. All efforts associated with Legislative Appropriations Request (LAR) funding must be evaluated by the agency and the QAT for consideration as a MIRP.

· Does the MIRP designation only apply to the development of new systems?

o No. MIRPs can include both the implementation of new solutions and large enhancements of an existing system. Determining whether your project meets MIRP qualifications is based largely on reviewing the project’s scope/schedule/budget as requested in the Legislative Appropriations Request (LAR). Please consult directly with the QAT for more specifics.

· If an agency follows an Agile DevOps methodology, then all work done on that application should be considered maintenance, not development. Does this work not qualify as a MIRP?

o While DIR and QAT support agencies implementing Agile DevOps methodologies, this approach doesn’t enable an agency to bypass statutory requirements for IT Project and MIRP reporting as detailed in TGC 2054.

Bottom line, if you are making new changes to an application that amount to greater than 80 hours of work, that is considered software development regardless of the team it is assigned to, and is subject to TGC, Chapter 2054; GAA, Article IX; and TAC, Chapter 216, requirements for IT Project and MIRP reporting.

· We recently closed a MIRP, and now are starting a new project to enhance the existing system. Do we need to report this to QAT?

o Any enhancement effort over 80 hours that is also over $5 million in total project costs must be reported as a MIRP.

· How does QAT Identify MIRPs in an agency?

o To perform its general duties, QAT maintains a portfolio of the state’s MIRPs. A MIRP is added to the portfolio when QAT receives information that a state agency intends to pursue such a project, and a project is removed from the portfolio when the project either submits its Post-Implementation Review of Business Outcomes (PIRBO) or is cancelled. Although QAT may identify projects for inclusion in the portfolio through multiple methods, the QAT’s primary source of information used to add projects to the portfolio is a state agency’s Biennial Operating Plan (BOP), including BOP amendments.

· What is the State Auditor’s Office (SAO) role for MIRPs?

o The Information Resources Management Act specifies that SAO serves on the QAT in an advisory role (Texas Government Code, Section 2054.158(c)). SAO’s appointed representative provides QAT input from an independent advisory perspective. SAO may perform non-audit service projects at the request and on behalf of QAT. A project is initiated by QAT when project performance, project completion, or other issues are identified during monitoring. Additionally, in accordance with Texas Government Code, Section 2261.258, the SAO annually assigns one of the following ratings to each of the 25 largest state agencies in that state fiscal year as determined by the Legislative Budget Board: (1) additional monitoring warranted; (2) no additional monitoring warranted; or (3) reduced monitoring warranted. Texas Government Code, Section 2054.1181, requires DIR to provide “additional oversight services” for all major information resources projects within agencies designated by SAO as “additional monitoring warranted.” Also, SAO, at the request of QAT, may conduct a non-audit service project of state agency major information resources projects. A project is initiated by QAT when project performance, project completion, or other issues are identified during monitoring. See SAO Project Reviews in QAT Policy and Procedures Manual for more details on the process.

· Is there a time/duration limitation (or maximum time frame) for MIRPs?

o No. However, the time frame/duration of the MIRP work typically should align with the appropriation funding the work, which is generally a biennium, or two years. For projects that will take multiple biennia to complete, the recommendation is to split that project into phases that align with the biennial appropriation. This approach is a demonstrated strategy for successful project delivery on time and within budget.

Step 2: Plan

· When does an agency submit its Business Case for a MIRP?

o An agency completes and submits to QAT and in LBB’s ABEST system a Business Case (including the Business Case Template, Business Case Workbook Template, and Statewide Impact Analysis Template) for a MIRP as part of its legislative appropriations request (LAR) prior to a legislative session and as part of its Biennial Operating Plan (BOP). If a project/initiative is identified outside of the regular LAR approval process, the agency must complete and submit a BOP Amendment to the Legislative Budget Board (LBB) and then complete a business case and submit it to the QAT. After Session, the Business Cases for funded projects are approved by the QAT, and letters are sent to agencies/IHEs with specifics on project start/end dates, approved budget, and monitoring requirements.

· Where do agencies submit Framework and other project-related documents (for example, contract drafts and monitoring reports)?

o Completed Framework documents must be filed electronically with QAT via the Statewide Project Automated Reporting (SPAR) system and by email at QAT@dir.texas.gov. For Monitoring Reports, agencies are also required to manually enter information from the Monitoring Report Template into the applicable SPAR fields. For information regarding SPAR, please contact QAT@dir.texas.gov.

· How do I determine my MIRP’s start date?

o From the Legislature’s perspective, the project starts in the planning phase, prior to any solicitation. Agencies are expected to have their Business Case Template, Business Case Workbook Template, and Statewide Impact Analysis Template Framework documents on file with the QAT at the time of solicitation posting(s). Typically, MIRPs included in an agency’s biennial Legislative Appropriations Request (LAR) begin September 1 following Biennial Operating Plan (BOP) approvals by the Legislative Budget Board (LBB). MIRPs identified outside of the LAR cycle should begin after a BOP Amendment is submitted to and approved by the LBB.

o For all MIRPs, the project start date is defined in the QAT approval letter, sent to the agency after the Business Case is submitted. A project “starts” as soon as resources (full-time equivalent (FTE) or staff augmentation) are assigned to complete tasks associated with the project planning, procurement, and/or implementation. Many times, the implementation start date is months or even years after the project’s actual start date.

· When can we start spending money?

o For projects included in the existing Biennial Operating plan, once the BOP is approved, agencies may spend up to 10% of appropriated funds starting September 1.

· I need to hire a contractor to complete the Framework Documents. Can I use project funds for this?

o If the project meets the criteria of a Major Information Resources Project (MIRP), approval by the Legislative Budget Board (LBB) of the Biennial Operating Plan (BOP) or a BOP Amendment is required before any project funds are spent, including on contractors. The project cannot begin spending funds prior to the appropriation’s effective date. Once BOP/BOP Amendments are approved, a project can only spend up to 10% of allocated funds prior to the QAT review of the project’s Business Case, Business Case Workbook, and Statewide Impact Analysis. In most cases, it’s the agency’s responsibility to complete the Business Case package, regardless of staffing.

· Who should approve/sign off on Framework documents?

o According to Texas Government Code, Section 2054.307, a state agency’s executive director, or the executive director’s designee, must approve all framework documents. Agencies may include additional signatories as desired.

· What milestones should be included in the Framework documents?

o Milestones should include major points during which deliverables will be completed. These include project start/end, Framework submissions, requirements gathering, design, iterations/sprints/releases, procurements (including required reviews), testing, implementation/go live, security, authorizations, stabilization/hyper care periods, and closing activities (including the required submission of the Post-Implementation Review of Business Outcomes (PIRBO) six months after the project ends.

· My project follows the Agile methodology. How do I enter milestones in Framework documents?

o For projects following an Agile methodology, the stages of the project do not adhere to sequential phases the same way a traditional waterfall approach does. An Agile approach allows for an indefinite number of incremented releases before the overall development effort is deemed completed. Include important tasks associated with deliverables, similar to milestones/schedules for Waterfall projects. Examples could include identifying epics, increments of sprints, feature/product completions, or business milestones, performance indicators, in addition to typical IT project management milestones.  The QAT is always available to discuss options for reporting progress; please contact us to discuss specifics for your project.

· We are planning a vendor solution for this project, so any milestone targets are tentative based on the vendor selected and their project work plan. How do we handle this?

o The Framework documents are required to be completed using the best information the agency has at the time. While procuring a vendor can change some milestones, each agency is required to provide a plan for how the project will be managed, a high-level project approach, a procurement strategy, and project outcomes/objectives. QAT recommends planning your milestones based on what you know now and have dates by which key goals/targets must be accomplished to ensure a successful project implementation. Deferring responsibility for project planning/outcomes/schedule and management to the solution vendor is not aligned with statutory requirements for agencies.

· What are Informational Costs?

o These are staff/full-time equivalent and program area costs for project implementation. For example, designating an agency staff member as a project manager is considered an informational cost, whereas, designating a vendor/contractor as a project manager is considered a capital cost.  In general, if any staff are spending 50% or more of their work time on a project, this cost is considered an informational cost and part of the total project costs.  Please enter these estimated costs into the Business Case Workbook as directed.

· What’s the difference between a Business Case and a Project Plan?

o A Business Case is a document that outlines the justification for a project. It identifies the problem that the project aims to solve, the benefits that the project will deliver, and the costs associated with the project. A Project Plan, on the other hand, is a more detailed document that outlines the specific tasks that need to be completed and how they will be completed to deliver the project. It includes more specific milestones, budgets, and resource requirements. While a business case is focused on the “why” of a project, a project plan is focused on the “how”.

· For a project involving multiple agencies, does each agency submit a separate Business Case and Project Plan, or can they collaborate on the same one?

o QAT encourages agencies to work together on interagency projects and maintain one set of project Framework documentation.

Step 3: Execute

· We are ready to send our Request for Offer (RFO) for a MIRP to the Contract Advisory Team (CAT). Do we need to take anything into consideration prior to this submission?

o Solicitation documents should be sent to the CAT for review through the Procurement Oversight & Delegation (POD) web application. CAT reviews may take up to 30 days to complete. Even if the review period exceeds 30 days, the agency does not have the authority to issue the solicitation until it receives a letter from CAT containing recommendations. Publishing a solicitation without receiving a letter from CAT may result in an audit finding.

To avoid impeding the agency's procurement process, state agencies should submit their major contract solicitation document(s) after final agency approval or for some state agencies, after final executive-level approval, (final executive-level approval may be necessary for some major contract solicitation documents, especially for IT related solicitations - see DIR IT Project Delivery Framework).

CAT requires the complete solicitation package to be submitted. This includes the primary solicitation and any additional exhibits or attachments that will be advertised, as well as supporting documents (such as a proprietary purchase justification).

While the QAT works closely with the CAT regarding solicitation submissions for MIRPs, there is not a separate required review for QAT for solicitation documents required by statute.  If a solicitation package is submitted to CAT that appears to be a MIRP, CAT will reach out to QAT to confirm the required Framework Documents have been submitted.

· We have a project that involves multiple vendors and purchase orders (POs)/demands as part of its implementation. Does this separation of work mean the project is not considered a MIRP?

o No. The project is based on the amount appropriated for the described project in the Legislative Appropriations Request (LAR). A single MIRP may have multiple vendors/demands/Purchase Orders (POs). 

· What notifications are required for contracts?

o State agencies are required to notify QAT within 10 days of the solicitation and award of all contracts of any value pertaining to major information resources projects. First, a state agency is required to notify QAT when the agency advertises a request for proposal, request for offers, or other similar process common to participation in the competitive bidding processes of a major information resources project. The agency is also required to provide the requisition number at the time of notification (Eighty-eighth Legislature, GAA, 2024–25 Biennium, Article IX, Section 9.02(b)(2)). Finally, a state agency is required to notify QAT within 10 business days of when the agency awards a contract for any major information resources project (Eighty-eighth Legislature, GAA, 2024–25 Biennium, Article IX, Section 9.02(b)(3)).

· When can the agency begin negotiations with a vendor?

o For MIRP contracts, valued under $10 million, negotiation may begin at any time after notifying QAT. For contracts valued at $10 million or more, agencies may begin negotiations only after submitting a first draft contract to the QAT and receiving confirmation of receipt (Texas Government Code, Section 2054.160(a)(1)).

· When does QAT review contracts associated with MIRPs?

o QAT must review major information resources project contracts valued $10 million or more at two points before contract execution. QAT reviews contracts for consistency with the best practices established by the Texas Contract Management Guide and the State of Texas Procurement Manual, with the goal of minimizing contractual risk and ensuring project funds are spent consistent with legislative intent for appropriated funds. QAT may provide feedback on the contract document or may waive the contract review requirement at its discretion.

o Step 1. For each contract for the development or implementation of a major information resources project with a value of at least $10 million, a state agency must submit the first draft proposed terms of the contract to QAT for review prior to the start of negotiations. The agency may commence negotiations after submission of the contract draft to QAT (Texas Government Code, Section 2054.160(a)(1)).

o Step 2. For negotiated contracts from step 1 above, a state agency must also submit the final negotiated, unsigned contract to QAT for review (Texas Government Code, Section 2054.160(a)(2)). QAT will review and provide recommendations on the final unsigned negotiated contract (Texas Government Code, Section 2054.158(b)(4)). The QAT makes every effort to provide comments by 30 business days after submission by an agency.  This process cannot be expedited. Submitting the final draft in redline, indicating changes from the original draft helps the QAT ensure it can complete its review within 30 business days.  There is no statutory requirement for the QAT to complete the review within any defined time period.  Agencies are required to develop procurement timelines including all required reviews and the full number of days for review as part of their Acquisition Plan.

After receiving the QAT recommendations, the agency must submit a response that details the final determination of each QAT recommendation. The agency will either indicate acceptance of recommendation(s) or provide explanation why the recommendation is not applicable to the project (Texas Government Code, Section 2054.160(b)). The agency must provide this required response to QAT comments prior to final contract execution.

Finally, a state agency is required to notify QAT within 10 business days of when the agency awards a contract for any major information resources project, regardless of value (Eighty-eighth Legislature, GAA, 2024-25 Biennium, Article IX, Section 9.02(b)(3)).

· If the contract for an agency project is not valued at $5 million, but the total project cost and amount appropriated is over $5 million, is the project still considered a MIRP?

o Yes, the dollar value threshold ($5 million) for determining a MIRP applies to the total project cost, including the contract.

· What does an agency do if a MIRP contract is amended during implementation?

o If an amendment causes a particular MIRP contract valued under $10 million to exceed the contract amount by 10% or more, you must complete and submit a Contract Amendment and Change Order Approval (CACOA) Template to document the changes in scope or cost and justify the need for the change. The form and draft amendment must be submitted to the QAT prior to executing the contract amendment.

o If cumulative overages cause the project to exceed current total project costs (including contractor, hardware, software licenses, full-time equivalents, etc.) by more than 10%, the agency must also revise and resubmit the project’s Business Case Template and Business Case Workbook Template from Step 1: Initiate. The revised Business Case and Workbook should capture the current progress of the project, the circumstances necessitating the changes, and all scope, budget, and schedule impacts of the change. Please also include all project change documents approved by the agency impacting this action.

o MIRP-related contract changes must be reported in the QAT Monitoring Report.

Step 4: Monitor & Control

· How do I update my project scope, schedule, and/or cost when there are changes?

o Follow your agency’s change management process (as defined in the required Project Plan that is completed and submitted for MIRPs) to document the change. In addition, note and describe the change in Section 6 Project Changes of the next Monitoring Report and in the Project Changes field in the DIR SPAR system following the change. Update applicable fields in both, as needed (do not change initial start date and cost). If the change is an increase of 10% or more, a revised Business Case Template and Business Case Workbook Template are required to be submitted to QAT. If the change is an increase of 50% or more, a cost-benefit analysis is required to be submitted to QAT.

· What is additional monitoring?

o In accordance with Texas Government Code, Section 2261.258, the State Auditor’s Office (SAO) annually assigns contract monitoring ratings of additional monitoring warranted, reduced monitoring warranted, or no additional monitoring warranted to each of the 25 largest state agencies. The Legislative Budget Board determines those 25 agencies. That statute requires a report on those ratings to be submitted to the Office of the Comptroller of Public Accounts (Comptroller’s Office) and the Department of Information Resources (Department) by September 1 of each year. The Comptroller’s Office and DIR are responsible for developing guidelines for additional monitoring. Texas Government Code, Section 2054.1181, requires DIR to provide “additional oversight services” for all major information resources projects within agencies designated by SAO as “additional monitoring warranted.” Below is an overview of the process for determining Additional Oversight services for agencies or institutions of higher education designated for Additional Monitoring by the SAO.

· QAT and DIR Additional Monitoring Determination Process    

The QAT has defined the following project performance indicators (for Cost, Schedule, Scope, and Quality),
to be submitted with the project’s approved monitoring report:

· Determination Process

QAT evaluates, based on any project within the “Additional Monitoring Warranted” agency, with the following criteria met:

Project evaluations consider all factors to determine true risk.

QAT will notify an agency of its additional oversight status through email.
The following image shows the QAT approach to Additional Oversight:

All new projects (not currently in flight) start at the low level of additional monitoring. If an agency is removed from the SAO additional monitoring list, and the agency is currently in the low level of additional monitoring, the project can return to quarterly monitoring as directed by QAT.

Projects assigned medium additional monitoring will be at that level for at least a one-year period and have the opportunity to request moving to a low level if project conditions warrant after the year of medium level additional monitoring.

Projects assigned high additional monitoring must maintain any procured services (for example, independent verification and validation (IV&V), quality assurance, additional project management) for the duration of the project.

If, under the additional monitoring, a state agency is assigned a status of additional monitoring or reduced monitoring, then the state agency shall follow the applicable guidelines. The SAO determines the contract period during which additional or reduced monitoring will occur.

· If QAT designates my project as needing a high level of additional monitoring, what happens?

o QAT will notify an agency of its additional oversight status through email. In addition to the required Acquisition Plan, Monthly Monitoring Reports, and Contract Closeout Report at contract termination, QAT may require the project to engage a quality assurance/independent testing vendor(s), independent verification and validation (IV&V) services, or additional project management. Projects assigned a high level of additional monitoring must maintain any of those procured services for the duration of the project.

QAT Approach to Additional Oversight

· Do we need to obtain Independent Verification and Validation (IV&V) services for a MIRP?

o While IV&V services are not required for all MIRPs, it is recommended to obtain them for projects valued over $10 million. Agencies may be required to obtain and maintain IV&V services for any MIRP at QAT discretion.

· In the monitoring report, we have planned start and end dates for milestones that have changed during the process. How do we report this?

o Follow your change management process as outlined in your Project Plan to document any proposed change. Additionally, note the specific cost, schedule, and/or scope change made during the monitoring period in Section 6 Project Changes in the Monitoring Report Template and in SPAR.  Any changes to the current project planned start and end dates for any milestone requires formal documentation from the agency prior to applying the changes to the project’s reporting.

· How do we determine the quality metric for our project?

o Texas Government Code, Section 2054.159, mandates the QAT monitor and report on performance indicators (Schedule, Cost, Scope, and Quality) for the entire life cycle of each major information resources project (MIRP).
Those performance indicators must be publicly reported via the Legislative Budget Board’s dashboard. Each performance indicator is given one of three QAT-defined performance ratings: Green, Yellow, or Red (see the Monitoring Report Template, Section 1.2, for more details on all indicators).
A MIRP’s quality standards for each project phase must be defined in the required Project Plan Quality Register to determine the Quality metric. At a minimum, to obtain a Green indicator during a monitoring period, a MIRP must have a submitted Quality Register that defines how the agency will determine project quality (for example, user acceptance testing results) through all project phases. In addition, the agency must follow its defined Quality Register to determine the Quality indicator that’s reported on the public dashboard for each monitoring period.

· What is required for Monitoring Report indicators that are not Green (i.e., Yellow or Red)?

o Agencies must include an explanation/mitigation for Yellow and Red indicators to be published with LBB dashboard results.

· Do I report all project risks/issues each monitoring period?

o Only report the risks/issues affecting the project during the current monitoring period you are reporting on. Include mitigations and dates/time periods by which those mitigations must be addressed to avoid additional project impacts.  Future planned actions can be mentioned as mitigations, but they should not be reported as official changes prior to agency formal approval.

· How do I determine Schedule Performance Index (SPI)/Cost Performance Index (CPI) for my project during a monitoring period?

o SPAR auto calculates these indices based on entries in the cost and date fields.
SPI = (Earned Value (EV)) / (Planned Value (PV)).
CPI = (EV) / (Actual Cost (AC))
EV = % project complete in project plan X Current estimated total project cost.
PV = Total days duration X Current estimated project cost.

o QAT also recognizes the following two approaches for Agile SPI/CPI:

SPI = Completed Story Points / Planned Story Points, OR

SPI = (Actual Story Points/Sprint(s)) / (Estimated Story Points/Sprint(s))

CPI = Completed Features Value (to date) / Actual Costs to Date, OR

CPI = (Estimated Hours/Story Point) / (Actual Hours/Story Point)

o Any Agile methodology needs to be approved by QAT prior to implementation and documented in the project details in the Statewide Project Automated Reporting (SPAR) system. It is always the agency’s responsibility to ensure that metrics are reported using accurate and complete information that is supported by data. If agency performance reporting is determined to be provided using any methodology other than the QAT-defined values, the agency will be required to undertake Corrective Action.

Step 5: Close

· What is a Post-Implementation Review of Business Outcomes (PIRBO)?

o After completion of a major information resources project, a state agency is required to prepare a post-implementation evaluation report to determine whether the project met its planned objectives or other expectations (as defined in the Business Case and Workbook), include any lessons learned and reporting on what was done well in the project. The agency is required to provide this evaluation report to QAT (as well as the agency’s executive director) (Texas Government Code, Sections 2054.306 and 2054.1182). A state agency is required to use the Post-Implementation Review of Business Outcomes Template, provided by DIR, to comply with the above requirements. A state agency is required to submit the Post-Implementation Review of Business Outcomes to QAT within six months of project closure.

· We have a Post-Implementation Report on Business Outcomes (PIRBO) due for a project, but the Project Manager has left the project. How should we approach the PIRBO?

o Using the initial information from the Business Case and Project Plan documents, complete the initial information in the PIRBO Template. QAT recommends updating the most recent version of the project’s Business Case Workbook (BCW) to include actuals/expenditures with the assistance of your agency financials liaison to ensure accurate cost reporting post-closure. In addition, lessons learned should be documented throughout the project, and these can be referred to in the PIRBO. Review previous Monitoring Reports in SPAR to gain historical knowledge of the project that can be used in the PIRBO. A QAT recommendation would be to include the PIRBO as a required deliverable in their contract. Regardless of staffing, the agency maintains the responsibility to submit the PIRBO on time.

· When do I submit a Post-Implementation Review of Business Outcomes (PIRBO)?

o PIRBOs must be submitted within six months of the project end date. Including a PIRBO milestone in your Business Case helps ensure this requirement is met timely.

· What does an agency do if a project is cancelled?

o In the event a MIRP requires cancellation, the QAT requires the following actions to be taken to remove the project from the agency’s portfolio. Until all steps have been completed by the agency, the project will remain in the MIRP portfolio, and the agency will continue to be subject to all required project reporting and monitoring.

To initiate official project closure/portfolio removal, the agency must submit the following:

§ An official letter on agency letterhead from the Chief Information Officer (CIO) detailing the date by which the decision to cancel the project was made and the governing body that made the decision.

The letter also must include the following information:

§ The specific circumstances leading to the decision to cancel the project.

§ The date the project intends to complete closing activities.

§ The details regarding final QAT reporting requirements, disposition of remaining deliverables, a communications plan, contract cancellation actions, team members’ reassignments, project/contract documentation archival activities, and the lessons learned compilation. 

After reviewing the submitted letter, QAT will send a response confirming the project cancellation and removal of the project from the agency’s MIRP portfolio.