Unit 5 / 11

Policy Analysis and Impact Assessment

Gains:

  • It structures a policy problem with AI by framing it in a framework of definition, options, criteria and recommendations.
  • It makes overlooked disadvantaged groups and hidden costs visible by producing a stakeholder map and cost-benefit framework.
  • He knows that the analysis produced by AI is limited to data and framed by value, and makes the final decision with discretion.

The most mental job of public administration is to put the right policy in front of a problem: "How can we reduce the traffic congestion in the city center?", "What incentive will help with youth unemployment?", "What burden will this new regulation impose on tradesmen?" These questions are multi-stakeholder, multi-criteria decisions made under uncertainty, with no single correct answer. Policy analysis (identifying a public problem and presenting recommendations to the decision maker by systematically comparing options, their possible impacts, costs and benefits) and regulatory impact analysis (RIA - RIA; assessing in advance who, how and how much a new legislation will affect) are the backbone of these decisions. Here AI is a very powerful thought partner in generating options, listing stakeholders and impacts, making counter-arguments visible, and drafting the analysis. But the decision, the value judgment and the responsibility belong to the human being; AI multiplies the options, you choose.

Skeleton of policy analysis

Good policy analysis follows a certain logic, and AI works well to build this framework:

  1. Identify the problem. Symptom or root cause? Who, where and how much does it affect? By numbers.
  2. Set goals and criteria. How will we measure success (cost, access, equity, feasibility, duration)?
  3. Generate options. At least 3-4 realistic options, including "do nothing."
  4. Evaluate the effects of each option. To whom is it beneficial, to whom is it a burden? cost; risks; applicability; unwanted side effects.
  5. Compare and recommend. Tables according to criteria; Offer a reasoned suggestion but keep the alternatives.
  6. Monitoring and evaluation plan. What will we monitor once the policy comes into force, and when will we review it?

AI can generate drafts in each of these six steps. Its most valuable contribution is seen in step 4: the human mind tends to fall in love with its own proposition and ignore negative effects and alternatives; Ask AI “what are the strongest counterarguments to this option, who would it harm?” Asking: opens this blind spot.

Tip: Use the AI ​​as the "devil's advocate": give your preferred option and say "write down the 5 strongest objections that will bring this down." This hardens your analysis against auditing.

Data driven but limited by value

Policy decisions are value-laden as well as technical. “Where should we spend the budget?” In the question, AI can make cost-benefit calculations; but "which group's needs take priority?" It is a political/ethical choice and belongs to the elected or authorized decision maker. Be sure to also verify the numbers the AI ​​produces: the model may produce made-up numbers that “look reasonable” rather than real data.

Attention: Any number given by the AI, such as cost, population, rate, etc., is not included in the analysis without being confirmed with official statistics or institutional data. A convincing fake number puts an entire policy on the wrong footing.

three mini cases

Case 1 — The overlooked stakeholder. A municipality was considering the option of moving its marketplaces out of town. When they submitted a list of stakeholders to AI, one group emerged that had not been included in the plan: older shoppers with limited access to public transportation. This group was added to the impact analysis; The decision was revised with transportation support.

Case 2 — Fake number was caught. In an incentive analysis, YZ said "there are 120 thousand registered businesses in this sector." When the analyst looked at the official statistics, the number was ~74 thousand. Verification prevented overestimating the incentive budget by 60%.

Case 3 — Counterargument strengthened the analysis. Before a draft regulation went to parliament, the team asked AI for the "8 strongest objections to this regulation." Six of these were actually mentioned in the parliament; Since the team came prepared, the discussion was concluded in one session instead of two.

Four copyable templates

1) Policy option manufacturer:

Your role: senior policy analyst. Generate at least 4 realistic policy options for the following problem, including "do nothing." Give a one-sentence explanation for each option. Don't evaluate yet, just list the options. Mark where you make assumptions. PROBLEM: [describe the problem with numbers]

2) Multi-criteria impact assessment table:

Evaluate the following policy options in a table based on the following criteria: cost, feasibility, equity impact, duration, key risk. Write a short justification, not a rating. State that every number you use must be "confirmed with official data". Under the table, list separately the value judgments that the person should make the decision. OPTIONS: [list]

3) The lawyer of the devil (counter argument):

I advocate the following policy proposal. Do the opposite: write down the 6 strongest objections that would destroy this proposal, its weakest points, and possible undesirable side effects. Specify specifically who you may harm. Don't be gentle, be tough. SUGGESTION: [suggestion]

4) Regulatory impact analysis (RIA) framework:

Your role: regulatory impact analyst. Produce an RIA framework for the following draft legislation: (1) problem solved, (2) affected segments (citizens, tradesmen, institutions), (3) burdens and costs, (4) expected benefits, (5) alternatives, (6) monitoring indicators. Mark each numerical claim as "to be confirmed". DRAFT: [draft legislative summary]

Weak prompt / Strong prompt

Weak: "How do we solve the traffic problem?"

Strong: "Senior policy analyst in the role. Generate 5 options for city center traffic congestion, including 'do nothing'. Evaluate each in a table based on cost, feasibility, equity impact, duration and key risk measures; mark each number you use as 'must be confirmed by official data'. Then for the two options that look the strongest, write counter arguments that will undermine them. At the end, list separately the value judgments (whose priority, which group's burden) that leave the decision to us."

The difference: powerful prompt establishes option generation, multi-criteria evaluation, data validation, counterargument, and human-decision boundary in a single flow.

Contribution and limit of AI

step

AI contribution

human responsibility

Problem definition

Draft, data collection

Accurate data, root cause

Option generation

Extensive option list

Realism filter

Impact assessment

Table, counter argument

Number verification, comment

Suggestion

Draft justification

value judgment, decision

monitoring plan

Indicator recommendation

Approval, implementation

Stakeholder map and cost-benefit balance

A good policy analysis answers not only the question "what should be done" but also the question "who will be affected and how?" Creating a stakeholder (individual, group or institution that is affected by or affects a policy) map makes overlooked grievances and points of resistance visible in advance. AI can quickly generate a large list of stakeholders for a policy option and the potential gains/losses of each stakeholder; But you can filter the suitability of this list to the concrete event, local realities and political sensitivities. Likewise, cost-benefit analysis (comparing the monetary and non-monetary costs of an option versus its benefits) is a framework that AI can build the framework for, but you have to fill in the numbers with the organization's own data.

Mini case — silent group of victims. In one district, the market place was planned to be moved; The economic analysis was positive. The stakeholder map created with AI highlighted a group that was never mentioned in the plan: approximately 600 senior citizens who came to the market on foot could reach the new location in 40 minutes by public transportation. This finding prompted the inclusion of a service line requirement in the decision.

Mini case — hidden cost. In a cost-benefit outline of a digitalization project, AI counted only the software license as a cost. When we added the team, training, transitional productivity loss and call center load, the first year cost increased by 60 percent and the turnaround time increased from 1 year to 2.3 years — the decision became realistic accordingly.

Template generating stakeholder and cost-benefit framework:

Task: Produce two tables for the following policy option.Option: [policy description]Table 1 — Stakeholder map: stakeholder | way of influence | possible earnings | possible loss | level of resistance (low/medium/high).Table 2 — Cost-benefit framework: item | monetary? | estimated direction (+/−) | data source [TO BE FILLED].Rule: Number fictitious; Leave the value of monetary items blank, ask for their source. Specify disadvantaged groups that may be overlooked in a separate heading.

Caution: The stakeholder list that the AI ​​produces consists of general groups that seem “reasonable”; It often bypasses local, silent and unorganized groups. You're the one who knows the field — it's your job to complete the list.

Common mistakes

  • Trusting the AI's numbers. Verify every figure such as population, cost, rate, etc. from the official source.
  • Just analyze the preferred option. Also consider “doing nothing” and alternatives; Otherwise the analysis will be biased.
  • Delegating value judgment to AI. "Whose needs come first" is a political/ethical decision, not a technical one.
  • Skipping counter arguments. Don't go to the council/supervisor until you see the strongest objections to your own proposal.
  • Incomplete listing of stakeholders. Particularly question voiceless/vulnerable groups (elderly, disabled, rural).
  • Not putting a monitoring plan in place. Policy that is not measured cannot be evaluated; Define the indicators from the beginning.

In summary

In policy analysis, AI is a powerful thought partner that generates options, charts impacts, reminds stakeholders, and most importantly, generates arguments against your own proposal. But every number should be verified, value judgments should be left to people, and the final decision and responsibility should remain with the authorized decision maker. AI extends analysis; You are the one who deepens the decision and signs it.

Application task

Choose a real policy issue on your unit's agenda. Extract at least 4 options with the "Policy option generator", compare with the "Multi-criteria impact assessment table" and confirm at least one number generated by the AI ​​with the official source. Finally, apply the "Lawyer of Satan" template to your preferred option and note the strongest objection you find.

checklist

  • [ ] I considered at least 4 options, including "do nothing".
  • [ ] I verified every numerical claim from the official source.
  • [ ] I have listed all affected stakeholders, including vulnerable groups.
  • [ ] I produced arguments against my preferred option.
  • [ ] I reserved value judgments for human decision.
  • [ ] I defined monitoring and review indicators.