Tutorial on Python i18n: How to Use Gettext Python Module

The primary purpose of Python app localization is to provide content in different languages and in user-friendly formats. Depending on the settings and locale of the user, they’ll see different languages ​​visiting the same application.

To translate Python apps you need to consider both internationalization and localization. These terms are often misused, therefore, let’s immediately agree that internationalization (i18n) will refer to the process of preparing programs for translation carried out by developers. Whereas, localization (l10n) is performed mainly by localization managers and translation teams, and is the process of translating and creating content for local audiences. In this article, we’ll focus mainly on python i18n.

Before translators can get to work and translate texts from one language to another, developers should go over all the locations containing text for the user (i.e. is part of the visual interface of the program) and make sure it’s suitable for translation. To achieve this we’ll be using the GNU gettext module.

What Is the GNU Gettext Module

The GNU gettext package is standard for implementing multilingual programs and gives us the gettext utility for processing translation files. It includes the xgettext program, capable of creating a template file for translation from sources not only in Python, but for a large number of programming languages as well, including C, C++, C#, Java, and more.

To proceed with this tutorial you’ll need to install the gnu gettext package if it’s not already installed. As some operating systems have it installed by default. Python has a standard pygettext module for this. This module is a wrapper of the standard GNU gettext module.

How the GNU Gettext Module for Python Works

Our task here is to have a simple program with different languages. Further in this article, we’ll show you how to translate the Python app we created for this demo. But before we go into the details, let’s take a quick look at a typical workflow when using gettext:

1. Wrap source strings you need to translate with the gettext function. For details and other functions see the official docs.

2. Run the default python pygettext.py module to extract translation strings from source code and generate a .pot (Portable Object Template) file. This file will hold the strings for translation and empty translations. Here’s an example:

#: demo_translate.py:12
msgid "Hello World"
msgstr ""

1. Copy the .pot template to a language-specific .po file, and translate your texts.
2. Compile the above .po file to a special .mo file to be used by our application. You can use the default python module msgmt.py to do this.

Let’s Translate a Python Demo Application

We created a demo application, so you can see how to implement the steps we mentioned above in more detail.

Creating a Demo Python Application

Let’s start by writing a basic program demo_translate.py:

import gettext

def main():
    for language in ["uk", "en"]:
        language_translations = gettext.translation("base", "locales",      
languages=[language])
        language_translations.install()
        
        _ = language_translations.gettext
        print(_("Hello world"))

if __name__ == '__main__':
    main()

In this app, we have two languages: English and Ukrainian. You can add as many or as few languages as you wish, all you need is to specify the proper language codes.

As you can see, we imported gettext under the name “_”. This, firstly, will reduce

the amount of code to call this function. Secondly, it will allow tools collection of strings for translation automatically organize translation files for us. These tools will run through all Python modules and select all of them calling functions named “_”. Also, other developers will immediately understand how to deal with the translation function.

Later in the code, we called the “_” function and passed it the “Hello World” string. This function immediately translates the transmitted string into the required language. Of course, it will still be necessary to organize a file with translations, but more on that later.

Translation files structure

The files are structured within subfolders – one per each locale. In our example we use the two letters code to represent the locales, but you can also use three-letters code and other variants, especially if you’re translating into dialects of the same language, as you’ll need to keep them separate. Here’s a list of the default language codes used in Crowdin, but you can also add custom languages and set up your own rules.

Inside each of these folders is another level of subfolders. All of them are called LC_MESSAGES. This is a generally accepted structure for storing translations. And already in each of the LC_MESSAGES folders, we have files that contain translated texts.

Base on our project settings, we’ll have two language folders:

• en: for English.
• uk: for Ukrainian.

Here’s an example of the file structure:

locales/
├── en
│   └── LC_MESSAGES
│       └── base.po
└── uk
    └── LC_MESSAGES
        └── base.po

Translation files here have the “.po” extension, so each language has its own .po files. You can also notice that some files can have the “.mo” extension. This is compiled binary result that is directly used in the translation of the text. Later in this article, we’ll learn how to create them.

To get the base.po files: we’ll need to create a ** base.pot** file first. We can generate a base.pot file in the locales folder taken from our demo_translate.py program. Remember that POT files are just templates and we should not edit them. This file is needed so translators can translate the relevant texts into a specific language after we copy this file into the folders.

To create it, execute this command:

$ pygettext -d base -o ./locales/base.pot demo_translate.py

If the console doesn’t find the pygettext file, you’ll need to find this file in your OS. The location of those files depends mainly on the Python library used on your OS. When you find it, try to use the absolute path.

When you successfully run the previous command. You will see the content in the middle of the file base.pot.

We have text templates in the header:

• Last-Translator - The name and email address of the translator who was the last to edit the translation.
• Language-Team - This field should contain the mailing list on which the translation team can be reached.

If you have more details about the header structure of this file you can read this link.

Here’s an example of the header:

# SOME DESCRIPTIVE TITLE.
# Copyright (C) YEAR ORGANIZATION
# FIRST AUTHOR <EMAIL@ADDRESS>, YEAR.
#
msgid ""
msgstr ""
"Project-Id-Version: PACKAGE VERSION\n"
"POT-Creation-Date: 2022-08-09 11:19+0300\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
"Language-Team: LANGUAGE <LL@li.org>\n"\
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: ENCODING\n"
"Generated-By: pygettext.py 1.5\n"

Now, we need to copy this file to dirs LS_MESSAGES and change the file extension to “*.po” . Take a look at the structure we already presented above:

locales/
├── en
│ └── LC_MESSAGES
│ └── base.po
└── uk
└── LC_MESSAGES
└── base.po

Translate Python app

In the previous paragraphs we already created dir locales. There are two dirs: en, uk. In each folder there is a LC_MESSAGES/base.po file.

All we have to do is translate our files into the language we specified. Remember we said earlier that English is our source language. Let’s translate it into Ukrainian. Translation will be added to the uk\LS_MESSAGES\base.po file.

Here’s what we’ll receive:

#: demo_translate.py:11
msgid "Hello, world"
msgstr "Привіт світ"

This kind of files is divided into groups, where each group represents one translation and is separated from other groups by one empty line. Each of translation strings is represented by the following lines:

• msgid: identifier of the translation string. It can’t be edited manually. It’s the source string wrapped in _(). • msgstr: translation into the language for which the current “.po” file is intended.

But “.po” translation files are not used directly. They need to be compiled first. The compiled versions of the translation files have the extension “.mo” and are compiled using another python msgmt.py module. Let’s combine our files with “.po” For English and Ukrainian language:

$msgmt -o ./locales/en/LC_MESSAGES/base.po ./locales/en/LC_MESSAGES/base

$msgmt -o ./locales/uk/LC_MESSAGES/base.po ./locales/uk/LC_MESSAGES/base

Once you complete this stepm you’ll see your “.mo” files:

locales
├── en
│   └── LC_MESSAGES
│       ├── base.mo
│       └── base.po
├── uk
│   └── LC_MESSAGES
│       ├── base.mo
│       └── base.po
└── base.pot

If you completed all the steps, you can run our program:

$python demo_translate.py 
Hello World
Привіт світ

This kind of files is divided into groups, where each group represents one translation and is separated from other groups by one empty line. Each of translation strings is represented by the following lines:

• msgid: identifier of the translation string. It can’t be edited manually. It’s the source string wrapped in _(). • msgstr: translation into the language for which the current “.po” file is intended.

But “.po” translation files are not used directly. They need to be compiled first. The compiled versions of the translation files have the extension “.mo” and are compiled using another python msgmt.py module. Let’s combine our files with “.po” For English and Ukrainian language:

$msgmt -o ./locales/en/LC_MESSAGES/base.po ./locales/en/LC_MESSAGES/base

$msgmt -o ./locales/uk/LC_MESSAGES/base.po ./locales/uk/LC_MESSAGES/base

Once you complete this stepm you’ll see your “.mo” files:

locales
├── en
│   └── LC_MESSAGES
│       ├── base.mo
│       └── base.po
├── uk
│   └── LC_MESSAGES
│       ├── base.mo
│       └── base.po
└── base.pot

If you completed all the steps, you can run our program:

$python demo_translate.py 
Hello World
Привіт світ

Akso, one of the popular Python web frameworks is Django. If you want to learn more about how to translate your Django web application, you can find more information in our Django i18n article.

Conclusions

Now you know all the basics of working with Python modules for translating texts from theory to practice. So go ahead and translate Python apps following the steps we recommended.