How OpenAI added a splash of colour to the Beige Book

If you’re an economist like me, you know how time-consuming it can be to read the Beige Book, the Federal Reserve’s summary of regional economic conditions in the US. It’s full of facts, numbers and vague anecdotes. But what if there were a way to bring it to life?

I used a form of AI called OpenAI, a powerful natural language model, to help me digest the Beige Book in a fraction of the time. OpenAI can summarise and generate questions and answers based on any text you feed it. It’s like having a smart colleague who can make sense of any document. Here's how I used OpenAI to add some colour to the Beige Book.

I provided OpenAI with a prompt to score text taken from a sample of 21 separate editions of the Beige Book. I then asked it to score the text on a numerical scale from one to ten, focussing on sentiment related to economic growth and activity. To avoid AI hallucination, which is when the model generates output not based on facts, I followed some common best practices.

First, I made sure my prompt was clear and specific. Second, I asked OpenAI to provide reasoning for its score, which I then verified against the Beige Book. Third, I always included reference text as a neutral baseline for comparison. Lastly, I made sure to be explicit in telling OpenAI to not make anything up! My results are show in the chart below.

The sample covers 1972 to 2023. The first thing to notice is the positive association between US GDP growth and our AI-derived value for Beige Book sentiment – meaning both variables tend to move in the same direction.

For example, during notable business cycle downturns as highlighted on the bottom left of the chart, both GDP and sentiment tend to be low or negative. Similarly, at pivotal turning points after recessions, the kickstart in GDP growth is typically accompanied by greater optimism in the tone and language of the Beige Book, such as in 1983 highlighted in the top right.

Many consider the Beige Book to be a respectable contemporaneous indicator for US economic activity. Indeed, it acts as a useful cross-check against erratic swings in quarterly GDP data, which is why we have plotted the data against the smoother ‘year-over-year’ (YOY) change. However, Covid was a sudden shock, followed by an equally abrupt recovery. So the 2020 Q3 Beige Book we fed to OpenAI captures the quarterly rebound relatively well, despite it appearing as an outlier as year-over-year GDP growth was still negative.

The latest edition of the Beige Book was released on 18 October 2023. The OpenAI sentiment score for this report is highlighted on the chart by the yellow star. This suggests a considerably weaker outlook for US GDP growth. According to the line of best fit, this could be somewhere close to zero percent growth. Or course, there is considerable uncertainty in this estimate, and we must wait for the magnitude of any slowdown to be revealed in the hard data.

Using OpenAI has helped me gain new insight from a large body of text like the Beige Book which contains a lot of data and information. OpenAI scanned portions of the document and extracted a summary and sentiment score in a matter of minutes. However, the tool is not perfect as OpenAI sometimes makes mistakes.

It might miss important details or nuance, and it may create output that is irrelevant or completely made up. I also noticed that OpenAI can often provide different answers for the exact same prompt. Therefore, best practice also involves verifying and evaluating output based on your own knowledge and judgement.

This is just one example of how AI is beginning to change how we work here at LGIM. And in case you were wondering, yes, I also used OpenAI to help me draft this blog post…