The Bank That Speaks Your Language: Analysis of the AI ​​Model from PKO BP

In an unprecedented move for the Polish financial sector, PKO Bank Polski has made its advanced language model available to the world. This is not just a technological breakthrough, but a strategic gambit that aims to redefine customer relationships and attract the best talent.

What will you find in the article?

1. What is Polish RoBERTa 8k from PKO BP?
2. Why RoBERTa and not BERT? Key differences
3. How does the bank use AI in practice?
4. Openness Strategy: Why does the bank share its technology?
5. Summary: A new standard in banking

When we think about innovations in banking, mobile applications and payment systems come to mind. However, PKO Bank Polski decided to go a step further, entering the territory previously reserved for technology giants. The bank created and then made publicly available on the Hugging Face platform its own AI language model - polish-roberta-8k. This is the first such move in the Polish financial sector and a signal of a fundamental change in the approach to technology.

1. What is Polish RoBERTa 8k from PKO BP?

To put it simply, it is an advanced artificial intelligence system, trained specifically to understand the nuances of the Polish language. It is not a generative model like ChatGPT, which creates new texts. Its main strength is **understanding and analysis** of existing content. It is built on the RoBERT architecture, which is an improved version of Google's groundbreaking BERT model.

The model was "fed" with a huge amount of Polish-language texts to learn grammar, syntax, context and meaning. Thanks to this, it can perform complex natural language processing (NLP) tasks.

PKO Bank Polski was the first financial institution in Poland to join the global Hugging Face community, making its AI model available.

2. Why RoBERTa and not BERT? Key differences

The choice of RoBERT architecture was not accidental. This is a conscious decision to choose a more modern and powerful variant. Although both models are based on the same concept (Transformer), RoBERTa (Robustly Optimized BERT Approach) introduces key improvements:

• More training data: RoBERTa is trained on much larger datasets (in the original version 160 GB vs 16 GB for BERT), which allows it to gain a deeper understanding of the language.
• Dynamic masking: Unlike the static “masking” of words in BERT, RoBERTa does it dynamically, which makes the learning process more diverse and effective.
• No NSP task: RoBERTa dispenses with the task of Next Sentence Prediction, which turned out to be less effective in BERT, focusing all its power on context understanding.

To put it simply: RoBERTa is like "BERT on steroids" - more powerful, trained on more data and optimized for deeper text understanding.

Characteristic	BERT	RoBERTa (used by PKO BP)
Masking strategy	Static	Dynamic
The size of the training data	Smaller (approx. 16 GB)	Much larger (approx. 160 GB)
Additional training task	Predicting the next sentence	None (focus on masking)

3. How does the bank use AI in practice?

PKO BP did not create a model for the experiment itself. It is already actively used to improve real business processes and improve customer relationships. The main applications are:

• Sentiment analysis: The model can automatically analyze thousands of opinions, comments and messages from customers to understand their moods, identify problems and assess satisfaction in real time.
• Classification of documents and emails: The system automatically categorizes incoming documents and messages, directing them to the appropriate departments. This significantly speeds up the flow of information and response time.
• Improving chatbots and voicebots: Thanks to a better understanding of customer intentions, banking bots can provide more precise answers and solve problems more efficiently, without the need to engage a consultant.
• Offer personalization: By analyzing communication with the customer (while maintaining full confidentiality), AI helps to better understand his needs and adjust the offer, e.g. by offering a loan to a company that mentions development plans.

4. Open Strategy: Why does the bank share its technology?

Providing such an advanced tool for free may seem surprising, but it is a well-thought-out strategic move. By publishing its model on Hugging Face, PKO BP achieves several goals at once:

• Employer Branding: It positions itself as an innovative technology leader, not just a conservative financial institution. It is a powerful magnet for the best talents in the IT and AI industries.
• Building an ecosystem: The bank encourages developers, startups and scientists to experiment with its model. This may lead to the creation of new, innovative solutions that PKO BP will be able to implement in the future.
• Contribution to the development of Polish AI: Providing a high-quality model trained on Polish is a huge contribution to the entire national technology environment.

5. Summary: A new standard in banking

The decision of PKO Bank Polski to create and share its own AI language model is much more than a technological curiosity. This is a signal that artificial intelligence is becoming a key element of the strategy in the financial sector, and the fight for customers is shifting to the field of deep understanding of their needs. By making its tool available, PKO BP is not only ahead of the competition, but also sets a new standard of openness and innovation for the entire industry.