AI vs. Health Economists: Can ChatGPT Build Budget Impact Models?

Following our experiment with AI-generated healthcare content, we wanted to test generative AI on a more quantitative task: could ChatGPT guide us through building a pharmaceutical budget impact model (BIM) in Microsoft Excel?

A BIM estimates the financial consequences of introducing a new intervention into a healthcare system compared with the existing standard of care. Even relatively simple models require a coherent structure, linked inputs and calculations, and careful validation.

Building the model with ChatGPT

We set out to create a simple three-year UK model comparing a new drug with one standard-of-care competitor. The model needed to include treatment, administration, monitoring and adverse-event costs.

We deliberately asked ChatGPT to lead the process in detail. Starting from a blank workbook, we requested step-by-step instructions for the workbook structure and then followed the AI’s guidance through inputs, calculations and results.

The system was surprisingly capable of maintaining context across the exercise. It could suggest worksheet structures, formulas, summary outputs and charts, and it adapted when assumptions were changed or additional inputs were introduced.

Where the weaknesses appeared

The experiment also highlighted why model-building expertise still matters. Some formulas required correction, and the limitations became clearer as we moved beyond the simplest calculations. Sensitivity analysis was particularly challenging: although we eventually produced a basic one-way analysis, the process required repeated intervention and the result was below the standard expected for a professional model.

A real-world BIM is also likely to be significantly more complicated than our test. Multiple comparators, treatment discontinuation, changing market shares, disaggregated costs, time-dependent assumptions and robust sensitivity analysis all increase the need for careful conceptualisation and quality assurance.

AI as a collaborator rather than a replacement

The result was a basic but functioning model. That was impressive for an early general-purpose AI system, but the AI did not independently deliver a validated health-economic model from start to finish. Human judgement was required to frame the problem, provide assumptions, identify errors and assess whether outputs were credible.

The most promising interpretation is therefore collaborative. AI may reduce time spent on model scaffolding, formula drafting, documentation and repetitive development tasks, while health economists remain responsible for model structure, evidence, validation and interpretation.

As generative AI becomes more deeply integrated into office and analytical software, the productivity opportunity is substantial. The challenge is to capture that opportunity without lowering the standards of transparency and validation expected in HEOR.

This article reflects the market and policy context at the date of publication.

Further reading

• Archived LinkedIn version of the original experiment