What’s New at Crowdin: October 2024

This month, we’re rolling out some powerful updates. From a revamped Context Harvester CLI with a context quality check to new Crowdin Store apps like Language Fallback Configurator and an AI debugging tool. For complex projects, we’ve introduced cross-review approval for tasks and improved savings visualization.

Plus, discover multilingual terminology extraction with AI that continuously refines your glossary across languages as your project is translated. One AI model builds a multilingual glossary, while another uses this glossary to pre-translate.

In October alone, our customers used over 10 billion AI tokens (11.6 billion, to be exact) when our team closed 517 tasks from the task manager and deployed 475 times. Enjoy checking out the latest features.

Crowdin Context Harvester Updates

Context Harvester CLI has undergone a major overhaul this month. We expect this tool to be one of the cornerstones of future UI localization projects.

crowdin-context-harvester check

Imagine you’re a translator working on a typing test app and come across the phrase “Activate blind mode.” How would you translate it? For most languages, a direct translation wouldn’t make sense without additional context. Especially when you learn that in the app, this isn’t a mode for visually impaired users; it’s a feature where errors are hidden to avoid distracting the user while typing. Without context, this phrase is highly likely to be mistranslated. Alternatively, if a translator decides to submit a “context request” in Crowdin, that could slow down localization (and create some work for the localization manager).

Now imagine that AI can look at your project strings before translation and tell which strings are impossible to translate or would likely get poor translations with the current context. How cool is that?

The new check command in the crowdin-context-harvester CLI does just that. It takes all the texts you have for translation, then takes their context and asks the AI if there is a risk of getting bad translations with the context provided. It also considers your target languages, as some target languages may require more context.

The command’s output is a CSV file that you or your team can use to improve the context.

UI localization projects are full of short texts like “enable blind more.” Without enough context, both humans and AI will struggle to produce quality translations.

The new check command, together with the improved extract commands, becomes a super powerful toolkit for building the best possible context for your UI localization project.

You can read more about the new command.

More AI Providers in the Context Harvester CLI

The #1 feature request we received was to support more AI backends in the tool. Previously, Context Harvester could use either OpenAI or Crowdin as its LLM provider. Now, it’s possible to use any of the most popular LLM providers, including Google Gemini, Microsoft Azure Open AI, Anthropic, and Mistral AI.

Improved LLM Integration in the Context Harvester CLI

Many modern LLM models can accept a lot of content in one request, e.g. the full source code of a typical mobile application, all in a single request. We improved the context harvester to make it the most efficient and use most of the modern context window size. The improvement resulted in better context extraction quality, especially on smaller codebases.

Real-time Multilingual Terminology Extraction with LLM

One of the complaints we’ve heard from linguists about post-editing is the recurring terminology errors made by the machine. MT or AI may simply ‘prefer’ some words over others, resulting in repetitive editing by the human.

This new feature would present each new, human-verified translation to LLM, asking it to extract terms for words that can be aligned between the source and translation of the segment.

The extracted terminology is immediately applied in the next translation request to the LLM (within the pre-translate prompt). All new terms are created as “drafts”, allowing you to decide what to do with the extracted terminology. For example, review and approve it for human use or keep it in a separate glossary for AI use only.

For example:

Source String: Create a user account

AI Translation: Créer un compte utilisateur

Human Edit: Créer un compte

(A draft term created: account -> compte)

Next Source String: Delete the user account

AI Translation: Supprimer le compte

(AI Adjusted: AI now incorporates compte in future translations.)

This format demonstrates how the AI learns from the human edits to refine its terminology usage.

The new feature is only available to Crowdin Enterprise customers at Business plans. We hope to have it available on other plans and on crowdin.com soon.

To use real-time multilingual terminology extraction with an LLM, navigate to your Crowdin Enterprise organization and AI > Prompts. Create a new prompt by selecting Alignment Settings > AI Alignment and then choose your prompt. Ensure your pre-translation prompt allows the AI to consider glossary terms.

For the best results, we recommend using the o1-preview model by OpenAI.

Simple Term Extractor: New AI Extraction Strategy

Simple Term Extractor, a quite often used Crowdin add-on for terminology extraction, previously offered two approaches:

1. Statistical (traditional) method: Identifies term candidates based on the frequency of words and expressions.

2. Machine Learning method.

A new third strategy (AI Extract) has been added that uses Large Language Models (LLMs).

Users with Simple Term Extractor will find a new “Prompts” tab where they can customize the extraction prompt. When configured, a new extraction strategy becomes available on the extraction screen.

Unlike the real-time multilingual term extraction that keeps translations consistent across all languages, the Simple Term Extractor focuses on one language at a time. This let us go into more detail for each term. Managers use it to build termbases that define terminology for both human and AI translators. You can use it alongside the Multilingual Terminology Extractor (a native feature available to Crowdin Enterprise customers starting from the Business plans) — first, let linguists build and describe your initial term list with the Simple Term Extractor app, then enable multilingual terminology extraction and let linguists add corresponding translations thus, automatically create terms.

Custom Workflow Steps API Module

Crowdin apps can now register as workflow steps in Crowdin Enterprise. This opens the door for many more apps to come in the coming months.

The first custom workflow step is a Delay app. The app implements a delay for strings in the workflow until they meet certain criteria, such as until the whole file is translated.

Task Management with Cross-Review Approvals

Crowdin now offers a Pending status for tasks, enabling a cross-review workflow before any work begins. This feature is ideal for companies requiring budget or management approval, as it allows one manager to create a task and another to review and approve it before a linguist can start working on it.

To enable the Pending status, go to Project Settings > Privacy & Collaboration > Tasks > Cross-review. This simplifies task oversight and ensures that only approved tasks move forward to the translation phase.

Crowdin Store

Continuous Fine-Tuning

Now thanks to this new app you can boost translation quality with ongoing machine translation training based on your own data. The Continuous Fine-Tuning app automatically fine-tunes your MT by leveraging up-to-date translations from your project, providing increased accuracy and customization to meet your specific needs.

Better Fallback

Crowdin’s native fallback mechanic is pretty simple — it defaults to the project’s source language. While there are tools to handle translations into multiple dialects, there hasn’t been a way to set up more complex fallbacks. For example, if you’re translating English into five Spanish dialects, and only one dialect gets a translation, there’s no way to apply that translation to all dialects on export.

Another challenge is when translating from German into English and French — by default, you can’t set English as a fallback instead of German, which is the source language.

The Better Fallback application addresses both of these issues. You can now configure your own default fallback language and set up a custom fallback chain. Fallbacks are particularly useful for UI localization projects.

Creatio Bulk Emails

Creatio is a no-code platform for building ERP, CRM, and various business tools. We’re announcing a new integration with Creatio Bulk Emails, which will allow our mutual customers to translate their marketing campaigns with Crowdin.

If you’re using a different marketing tool, feel free to explore our store for other connectors to make your marketing content multilingual.

OpenRouter

We are amazed at the speed at which the LLM ecosystem is growing. Crowdin tries to experiment with all the latest technologies and integrate them when we see the potential to improve our customers’ experience. The new OpenRouter integration is a new AI provider in the Crowdin store that allows many of the most popular LLM to be consumed through OpenRouter with some benefits like better performance tracking, LLM error handling and more.

AI Debug

There’s a lot of unpredictability when using LLMs for linguistic tasks. Unlike machine translation (MT), where results are consistently presented, LLMs can behave in ways that are hard to debug and explain.

The new AI Debug app is here to help you troubleshoot when things don’t seem to work as expected. It acts as a proxy, capturing both the request Crowdin sends to the LLM provider and the response it gets back.

With this information in hand, you can often pinpoint where things went wrong.

For more details on how to use it, check out the app description in the Crowdin Store. Keep in mind, that the AI Debug app works best when debugging AI providers compatible with the OpenAI API and may not be as effective with other AI providers.

Sanity

Sanity is one of the most frequently requested integrations at Crowdin. Tinloof, a boutique development team, has created a powerful integration for one of our mutual clients. While this integration isn’t easily reusable, Tinloof is happy to configure it for other interested clients.

Due to the flexible nature of the Sanity CMS, it’s challenging to build an elegant, one-size-fits-all solution. In this case, a customized approach might be the best option for those needing tailored solutions.

Please contact Tinloof to find out more about the integration.

Add File Context in the Editor

We’ve introduced a handy mini-feature. It’s now possible to add file context directly from the right-hand side of the Crowdin Editor (available to managers and developers).

Context can still be added from the Sources tab by selecting a file and going to Settings > Context, which is particularly useful for UI localization.

We recommend you try the AI-powered File Context Extractor, which works well for many content translation projects.

Visualizing Savings Through TM, MT, and AI

In your project, you can now access a new pie-chart showing how much you’ve saved by using technologies such as Translation Memory (TM), Machine Translation (MT), and Artificial Intelligence (AI) in your translations.

New Actions for Selected Translation Text

We’ve added new features for managing selected translation text in our editor. By highlighting text, you can easily search for it in TM, terminology, or Wikipedia, change the case, and edit the font (bold, italic, underline).

Looking ahead, we plan to enable various Crowdin apps and tools, including AI, to register their own menu items for actions on both source and translation texts. For example, take action on selected text using ChatGPT, such as paraphrasing or searching for synonyms, and more.

New QA Check for Number Consistency in Translations

We’ve added a new QA check for numbers in translations. Now, the system automatically verifies that all numbers in the source and translation match, preventing errors like “400 EUR” instead of “40 EUR.” This ensures higher accuracy in translations and helps avoid typos.

Role-Based Authentication Settings on Crowdin

To enhance security on Crowdin.com, we’ve refined our two-factor authentication (2FA) settings. Now, you can choose which roles---managers, developers, or all project members---are required to use 2FA for project access. Previously, if 2FA was enabled, it applied to everyone. Now, you have the flexibility to tailor this requirement to specific roles.

Xcstrings Editing, Xlsx Empty String Support, MDX Key Display, and More

Here’s a quick summary of our latest updates on handling various file formats:

• Editing source texts directly in Crowdin is a highly popular feature. In string-based projects, you can edit all texts, but in file-based projects, only certain file formats are editable. This month, we’ve expanded this functionality by adding the ability to edit texts from Apple’s new .xcstrings file format.

• It is now possible to import and export empty strings in .xlsx files.

• Improved context information for every segment in .md and .mdx files: Front matter keys in .md and .mdx files are now displayed as keys or included in the context log. This also contributes to the expected quality of translation with AI.

• Context editing is now disabled for .GETTEXT files.

API Updates

• When listing project strings, it’s now possible to filter them by taskId.

• We had a nasty bug that prevented API users from editing labels for certain file formats. It’s now fixed.

• The getProject API endpoint now provides more details for vendor organizations, allowing them to find which client the project came from.

• It’s now possible to delete translations and approvals in bulk via the API.

Crowdin Apps SDK [private beta]

We’re excited to announce the release of a private beta for our new SDK, designed to speed up Crowdin app development. If you’re interested in accessing the tools for building apps quickly, reach out to us for early access.

Not a developer but have a feature or improvement in mind for the Crowdin platform? We’d love to hear your ideas! Our team is always on the lookout for innovative app concepts to bring to life.

Other Small, But Handy Updates

• You can now specify the model name in the configuration prompt, even if the AI provider doesn’t list it as supported. This resolves an issue with some providers where fine-tuned models weren’t returned.
• Portuguese and Brazilian Portuguese are now supported as separate languages in the DeepL integration.
• Storyblok integration now allows you to configure which Spaces you work with. Useful improvement for customers with many spaces, where some spaces are not intended for translation.
• The Corrupted Characters Check is a new custom QA feature for Crowdin Enterprise, designed to ensure that invisible or corrupted characters don’t make it into production.
• We’ve introduced a Getting Started Wizard to make it easier to set up AI in Crowdin, guiding new users from the first click to seeing AI at work.

External Tools Updates in October

• Figma Plugin Version 77: Adds option to remove approvals or translations when editing strings
• .NET API Client 2.26.0
• JS API Client 1.37.0, 1.37.1, 1.38.0
• Java API Client 1.18.1
• CLI 4.3.0: New commands and options added for project creation and bundle customization, including multilingual export and source language inclusion. Enhancements include screenshot upload improvements and more. See the release notes for a complete changelog.
• VS Code plugin 2.2.3
• Android Studio plugin 2.1.0