CBLM：中央银行的语言模型（英）33份

Central Bank Language Models (CB-LMs): Enhancing Monetary Policy Analysis

Introduction

Communication has become a critical tool for central banks to manage public expectations. Leveraging natural language processing (NLP) techniques, there is a growing body of research applying language models to analyze central bank communications. However, most existing models are trained on general text corpora, which limits their ability to capture the nuances specific to central bank communication.

To address this, the paper introduces Central Bank Language Models (CB-LMs), which are specifically trained on a comprehensive corpus of central bank speeches, policy documents, and research papers. These models outperform foundational models in predicting masked words in central bank idioms and classifying monetary policy stance from Federal Open Market Committee (FOMC) statements. Larger generative language models (LLMs) perform well in more complex scenarios, such as sentiment classification of extensive news related to U.S. monetary policy.

Key Findings

1. Performance of CB-LMs:

   • CB-LMs outperform foundational models in predicting masked words in central bank idioms.
   • CB-LMs also surpass state-of-the-art generative LLMs in classifying monetary policy stance from FOMC statements.
   • Larger LLMs outperform domain-adapted encoder-only models in complex scenarios, but present significant challenges in terms of confidentiality, transparency, replicability, and cost-efficiency.

2. Comparative Analysis:

   • CB-LMs are developed by retraining prominent encoder-only models like BERT and RoBERTa on a large-scale central banking corpus.
   • The performance of different LLMs is assessed across diverse training approaches and downstream task scenarios, providing deeper insights for central bankers to make informed model selection decisions.

Methodology

• Data Collection: CB-LMs are trained on a comprehensive corpus of central bank speeches, policy documents, and research papers.
• Technical Steps: The models leverage prominent encoder-only language models, such as BERT and RoBERTa, and are retrained on the specified corpus.

Applications

• Masked Word Test: CB-LMs outperform foundational models in identifying common central bank idioms.
• Monetary Policy Sentiment Classification: CB-LMs effectively classify the stance in official monetary policy decision statements.

Comparative Analysis with State-of-the-Art LLMs

• State-of-the-Art LLMs: These models require less retraining on central banking corpora due to extensive pretraining on diverse datasets.
• Performance Analysis: The study evaluates the performance of CB-LMs against state-of-the-art generative LLMs in various scenarios.

Challenges and Considerations

• Confidentiality, Transparency, Replicability, and Cost-Efficiency: Larger LLMs pose significant challenges for central banks in terms of these factors.

Conclusion

The development and dissemination of high-performing CB-LMs open up new possibilities for more accurate and insightful analysis of monetary policy and related topics. The paper also explores the adaptability of different LLMs within the context of central banking, providing deeper insights for central bankers to make informed decisions when selecting models tailored to specific requirements and technical environments.