Rethinking monitoring and evaluation in Pakistan’s power sector

Monitoring and evaluation (M&E) is an essential part of public policy. Governments formulate policies, allocate resources and establish institutions to achieve certain objectives, but the effectiveness of these interventions can only be determined by examining whether they produce the intended results. This makes M&E more than an administrative exercise. It is a mechanism through which governments assess their own decisions, identify implementation failures and determine whether existing policies require adjustment. An important question, however, receives relatively little attention in Pakistan’s public administration: who should finance this function? Consider a situation where the Ministry of Energy requires institutions under its administrative control to contribute towards the expenses of monitoring and evaluating of their performance. On the face of it, it relates to the present situation where sectoral entities like the PITC and the PP&MC are required to be funded by the DISCOs etc. At first glance, such an arrangement may appear reasonable. The institutions being monitored are the immediate subjects of the exercise and may be considered appropriate sources of financing. Yet the arrangement raises a more fundamental question about the allocation of responsibilities between a ministry and the institutions it oversees. If monitoring and evaluation is undertaken to assess the effectiveness of government policy, should its cost be transferred to the institutions implementing that policy, or should it form part of the ministry’s own financial responsibility? The distinction matters because M&E operates at different levels of government. An electricity distribution company monitors its operational performance, including technical losses, billing efficiency, recoveries and service delivery. Such monitoring is an integral part of its management responsibilities and should ordinarily be financed through its operational budget. A ministry, on the other hand, evaluates whether the policies governing distribution companies are achieving their intended objectives. This is a different function, involving not merely the performance of individual institutions but the effectiveness of the policy framework within which they operate. The difficulty arises when these two functions are treated as interchangeable. Pakistan’s existing public financial management framework provides a useful starting point for examining this distinction. The Public Finance Management Act, 2019, establishes a framework for performance-based budgeting and requires government expenditure to be based on defined plans. It also provides for the inclusion of policy goals, outputs, outcomes and performance indicators in medium-term performance budgets. These provisions recognise that government expenditure must be connected to the results it is intended to achieve. The question, therefore, is not whether monitoring and evaluation should be undertaken. It is whether the financial arrangements supporting it are consistent with the responsibilities assigned to different institutions. This is particularly relevant to the power sector, where institutional performance is shaped by decisions taken at several levels. Distribution companies are responsible for operational management, but their performance is also influenced by tariff policy, investment decisions, subsidy arrangements, procurement, transmission constraints and government interventions — basically, to assure its socioeconomic considerations. Now it is important to understand that an evaluation confined to a company’s operational indicators may identify a performance deficiency without adequately explaining the conditions that produced it. Consider distribution losses. A company may be held accountable for losses exceeding prescribed benchmarks, but a comprehensive evaluation must also examine whether its management possesses the necessary operational authority, whether investment requirements have been met and whether enforcement arrangements are functioning. Additionally, the general writ of the government too needs to be considered as this aspect has serious ramifications on the DISCO operations. Similarly, transmission constraints cannot be evaluated solely by examining the performance of the transmission entity without considering planning, project approvals, financing and coordination across the sector. These are questions of policy effectiveness, not merely institutional compliance. This is where M&E assumes the character of a meta-policy function. It examines the relationship between policy objectives, institutional arrangements, implementation and outcomes. Its purpose is not simply to determine whether an institution has performed according to prescribed indicators, but also whether the government’s policy design and implementation arrangements are appropriate. When the ministry undertakes such an evaluation, it is effectively evaluating the performance of its own policy framework. The responsibility for financing that exercise should ordinarily follow the responsibility for conducting it. There is also a public finance dimension. Transferring ministry-level evaluation expenses to public enterprises does not necessarily reduce the cost of government. It may simply relocate that cost from one public budget to another. The ministry’s expenditure may appear lower, while the financial burden is reflected in the accounts of the institutions it oversees and in case of deeply regulated power sector, such expenses add on to the electricity tariff — to be eventually borne by the consumers. Such arrangements can make it difficult to determine the actual cost of policy administration and oversight. They may also complicate the assessment of institutional financial performance, particularly where companies are required to incur expenses over which they have limited control. In the power sector, as already discussed, this question becomes more consequential because expenditure incurred by electricity companies may, subject to regulatory approval and the applicable tariff framework, have implications for the costs ultimately borne by consumers. A ministry’s general oversight expenses should not become an indirect charge on electricity consumers merely because the institutions being evaluated operate within the electricity supply chain. The issue is not that every contribution by a public enterprise is inappropriate. Rather, any such contribution should have a clearly established purpose, an appropriate legal and financial basis, and an identifiable relationship with the institution’s own responsibilities. Another concern relates to the independence of evaluation. If an institution being evaluated is also required to finance the exercise, questions may arise about the selection of evaluators, the determination of their terms of reference (SLAs etc.) and the treatment of findings. These concerns do not automatically invalidate institution-funded evaluations. Independent audits and external assessments can be financed by the institutions concerned without necessarily compromising their credibility. What matters is the institutional arrangement governing the exercise. Who appoints the evaluator? Who determines the methodology? Who receives the report? Can the findings be altered or withheld? And who is responsible for acting upon the recommendations? The answers become particularly important when an evaluation is intended to inform government policy rather than assess a specific operational activity. Pakistan’s development planning framework already recognises different stages of monitoring and evaluation. The Planning Commission’s PC-III, PC-IV and PC-V arrangements address implementation progress, project completion and post-completion performance. The framework also assigns reporting responsibilities to agencies operating and maintaining completed projects. This demonstrates that M&E responsibilities can legitimately be distributed across institutions. It does not, however, follow that every institution should finance every evaluation commissioned by the ministry exercising administrative oversight over it. A distinction must also be maintained between attached departments, autonomous statutory bodies and government-owned companies. Their legal status, financial powers and budgetary arrangements differ. Whether a particular institution can be required to finance a ministry’s evaluation must therefore be examined against its enabling law, approved budget and applicable financial rules. Administrative control alone should not be treated as a sufficient explanation for every financial obligation. Experience in other common-law jurisdictions illustrates how these responsibilities can be distinguished. India offers a particularly relevant example. Its Development Monitoring and Evaluation Office (DMEO), an attached office under NITI Aayog, undertakes independent assessments of central government schemes and provides evaluation support to ministries and departments. DMEO states that it has separate budgetary allocations, dedicated manpower and functional autonomy. It has also commissioned sector-wide evaluations through external agencies, with findings shared with the Department of Expenditure and the ministries concerned. This arrangement treats independent policy evaluation as an identifiable public function with its own institutional capacity, rather than assuming that each implementing body must finance the central evaluator. India’s approach does not mean that every evaluation is paid for by NITI Aayog. The applicable funding arrangement depends on the scheme and the commissioning authority. Indeed, DMEO’s guidance expressly provides for financing monitoring and evaluation studies through consultancy arrangements. The relevant lesson for Pakistan is therefore more precise: evaluation responsibilities, funding and commissioning authority should be defined in advance, and independent evaluation should have a visible budget and governance structure. The United Kingdom draws a similar distinction between policy responsibility and evaluation delivery. Its Evaluation Task Force states that responsibility for evaluating a policy rests with the government department responsible for that policy, while departmental analytical leadership oversees evaluation priorities, staffing and budgets. The Treasury’s Magenta Book advises that evaluation resources, including internal expertise and external expenditure, should be built into the business case at an early stage. Evaluation is thus planned as part of a policy intervention, rather than treated as an unforeseen expense to be recovered from the organisations later examined. Australia’s Commonwealth Evaluation Policy likewise expects government entities and companies to integrate evaluation into programme governance, identify resources early and distinguish evaluations intended to improve programme administration from those informing policy decisions. It does not prescribe a single payer for every assessment. Taken together, these examples support a principle of institutional clarity, not an absolute prohibition on contributions by implementing agencies: the purpose, commissioner, financing and independence safeguards of an evaluation should be explicit. A more coherent approach would begin by identifying the purpose of the evaluation. Where monitoring concerns an institution’s routine operations, the institution should ordinarily provide the necessary resources but from within its HR/resources and as a regular feature through provision of data etc. through set returns etc. Where evaluation concerns an approved development project, its financing should follow the applicable project arrangements. Where a ministry commissions an assessment of sector-wide policy effectiveness, the expenditure should ordinarily be provided for through an appropriate government budget. As already explained in the preceding paras, this would not prevent ministries from obtaining information, requiring performance reports or undertaking independent verification. Nor would it relieve public enterprises of their obligation to maintain credible internal monitoring systems. It would simply establish a clearer relationship between the purpose of an evaluation, the authority commissioning it and the financial responsibility for undertaking it. The Power Division could also benefit from a more integrated evaluation framework that distinguishes operational reporting from policy assessment. Distribution companies, transmission entities and other sector institutions already generate substantial performance information. The value of ministry-level M&E should lie in bringing this information together, examining the relationships between institutional performance and government decisions, and identifying the changes required across the sector. Collecting additional reports or commissioning separate assessments will have limited value if their findings do not inform policy formulation, budgetary decisions and subsequent implementation — specially, when the power sector is heavily regulated. Perhaps the more important question for Pakistan’s power sector is whether government is evaluating institutions without adequately evaluating the policies under which those institutions operate. Performance indicators can identify deficiencies, but they cannot independently determine whether the underlying policy remains appropriate. That requires an assessment of objectives, incentives, institutional capacity and the distribution of responsibilities. Monitoring and evaluation must therefore be understood as a continuing part of the public policy cycle rather than an administrative activity undertaken after policy decisions have already been made. A distribution company may be held accountable for excessive losses, poor recoveries or delayed investment. But what happens when these outcomes are also influenced by government decisions concerning tariffs, subsidies, investment approvals or institutional management? If the ministry evaluates the performance of institutions under its control, who evaluates the ministry’s own policies and decisions that shape their performance? This is perhaps the more fundamental question of monitoring and evaluation. Accountability cannot operate in one direction alone. An effective evaluation framework must examine not only whether institutions have implemented government policy, but also whether the policy itself was appropriately designed, adequately resourced and capable of achieving its intended objectives. The financing of M&E may appear to be a relatively minor question within the larger challenges confronting Pakistan’s power sector. Yet it reflects a broader issue of institutional accountability. A government that formulates policy must also maintain the capacity to determine whether that policy is working. Where it commissions evaluations of its own policy framework, it should ordinarily accept the corresponding financial responsibility. The true test of monitoring and evaluation is not how many reports are produced, how frequently institutions are inspected or how much expenditure is allocated to oversight. It is whether the findings lead to better decisions, clearer institutional responsibilities and measurable improvements in outcomes. Until the authority to formulate policy, the responsibility to evaluate it and the resources required to undertake that evaluation are properly aligned, monitoring may continue to expand without necessarily improving the effectiveness of government.

Read full story on Business Recorder