In this article, I wanted to explore different text conventions: text that reads left-to-right (LTR) and text that reads right-to-left (RTL). More specifically, the odd pretzel-logic software makes when selecting and editing the text of both of these conventions simultaneously.

Combining left-to-right and right-to-left text together is called bidirectional text, or BiDi for short.

Different Text Directions

Before we start, I want to present a couple of sentences that will be difficult to read. If you can’t understand them, don’t worry because that’s the idea.

Matters order that. Below line next the on but left the from again reading continue we, reached is line a of end the when and, right to left from read is words of string a.

That’s gibberish, right? When we write down something and then later read that something, the order we read and write matters. For example, this sentence is written in English, which is read from left-to-right. The direction we read text is part of what helps us derive the meaning.

The term script is going to be used to refer to the writing system a language uses to visually record messages.

The direction that writing is intended to be read/written is called its directionality. There are a handful of directionalities that exist, but for this article, we’re mainly concerned with the two that are often seen in software: left-to-right and right-to-left, where both read lines from top to bottom.

• English, for example, uses a left-to-right writing system. Such as the text in this article. It’s read from left to right, top to bottom.
• There are also right-to-left writing systems. They start at the right and read by advancing in the left direction. They are also read from top to bottom.

The Example

For this article, we’re going to make up an example text. Imagine some guy is on a computer, writing a report in English, about another language that’s written RTL:

It could be said that the Arabic say “يقول الكتاب الإنجليز أننا نقول أشياء تبدأ بعبارة ‘يمكن أن يقال’ ، لكن هذا محل خلاف” but more evidence is needed to confirm this.

The non-English text in the example above is in Arabic. It was taken from Google Translate.

It’s not a well-written sentence by any measure for either language – it’s not meant to be. It’s just a baseline we can test and analyze.
For the record, I’m not familiar with Arabic or any other RTL language or script.

Visually Reading Nested BiDi

Something that really tripped me up was comprehending the nesting of one directionality inside of another. Especially when line breaks are involved, so let’s explore that.

Let’s look at non-BiDi reading first. Since this article is written in English, we’ll start with LTR directionality. In the diagram below, we will analyze how the eye is expected to travel through LTR text.

And here’s the legend for the diagram above – as well as for additional ones below:

I won’t explain the diagram because you literally had to perform that reading process to get this far in the article. So I’ll assume everyone understands it (famous last words?). A return sweep is the movement the eyes need to make at the end of a line of text to continue reading on the next line. A directionality switch sweep is a term I made up because I have no idea how even to begin searching for the actual word for this; it’s the context switch the reader needs to make to switch from reading one directionality to another.

Now let’s analyze RTL directionality:

There’s really nothing different about it except for the horizontal reading direction.

Now let’s say a bilingual scholar reads the (BiDi English/Arabic) report:

When they read the English part, they start reading from left to right. And when they reach the Arabic, they have to jump to the right of the line and read right to left. After they reach the very bottom left of the Arabic, they jump to the very right of the Arabic and continue reading the rest of the English from left to right. And all this while also considering line breaks. If that sounds convoluted and chaotic, I agree.

This jagged reading pattern is exhausting to think about, but perhaps not a big issue if you’re fluent in both languages. Also, a sane person might avoid the issue and separate them (the different directionalities) with linebreaks instead of making them share the same lines of text:

Byte Representation

Given the test string, with different directionalities, how is it represented in bytes? Let’s look at this to see how bytes are stored in saved text files and how this text is represented during runtime in memory.

For both display and byte representation, the string rules for directionality and BiDi are part of the Unicode Standard.

To look into this, with a hands-on approach, we’ll save our test string in humble ol’ Notepad. And then browse its contents in a hex editor (HxD).

In the illustration above, I’ve highlighted (in a red box) where the start of the Arabic touches the English because we’re going to focus on that text in the diagram below. You will notice the Arabic text in Notepad and the hex editor look different. This is because the editor’s “Decoded text” section shows individual (ANSI) bytes, and Notepad saves the characters encoded in UTF-8. An important feature of UTF-8 is that any character code that’s not originally an ASCII value (between 0 and 127) will take multiple bytes to represent.

For this example, we’re looking at a UTF-8 file, but note that other string types may have different byte patterns. But if the string is in a Unicode format, the order of how characters appear in memory will be the same.

The illustration above gets wild because multiple individual characters with their own glyphs can come together to make a single glyph – which is unwieldy for demonstration purposes but please bear with it. Notice that the bytes’ order for the Arabic characters is stored in reading order – i.e., how we would encounter the characters when reading from right to left.

Logical Order

To expand on that previous sentence, when representing bytes of text, both LTR and RTL text store their characters in reading order.

• So LTR text has their characters stored in the order they would be read: from left to right.
• And RTL text has their characters stored in the order they would be read: from right to left.

So for RTL text, the order (the byte positions) that the characters are stored in memory is opposite to how they’re displayed on the screen.

This is referred to as storing the string in logical order.

Text Software Issues

Displaying text in either directionality isn’t too big of an issue. It’s easy enough to display text going right-to-left as well as left-to-right. It’s also not that big of an issue to display both directionalities at the same time, in the same document – even if one is nested inside of another.

But, beyond simply displaying the text, there are some other eccentricities. Let’s take a look at some of them.

Text Selection

Let’s examine selecting text. Text selection for either directionality is easy enough to imagine. But what about selecting between BiDi text?

Here’s our sample text again.

It could be said that the Arabic say “يقول الكتاب الإنجليز أننا نقول أشياء تبدأ بعبارة ‘يمكن أن يقال’ ، لكن هذا محل خلاف” but more evidence is needed to confirm this.

Try selecting it in a few ways:

• Start the text selection from the very left, and drag the selection a little bit to the right, but don’t select into the Arabic.
• Select some of the Arabic text, but none of the English.
• Start the text selection from the very left (the English part) and slowly drag the selection to the right through the Arabic, and back to the English.
• Start the text selection in the English part to the right of the Arabic, and slowly drag the selection to the left to the very beginning.
• Start the text selection in the middle of the Arabic and drag the selection to the left English portion.
• Start the text selection in the middle of the Arabic and drag the selection to the right English portion.

The big observation is that when the selection transitions from one directionality to another, there are jarring inversions of what’s selected. If we start selecting from inside the Arabic and select into the English on the left, the computer’s logic is “from where the user started the selection, select all the previous text up to where the mouse cursor is”. This causes the selection of Arabic to invert because the previous text is on the right. And vice-versa if we start the selection in Arabic and drag it into the English on the right.

If we select the entire Arabic portion, we don’t have to worry about where to start or stop – the whole text is selected and it’s as simple as that.

Text Editing

Since we’re probing this thing for observations, why stop with selections? Let’s continue (our descent into complete and utter madness) and complicate things further by exploring what happens when editing the text for these different directionalities.

Reset

Direction Keys

Place the edit cursor on the very top-left of the editing area and hold down the right key. When the cursor reaches the Arabic, it will jump to the right and move left. Then, when the cursor reaches the very left of the Arabic, it will jump to the right. So this is pretty much the same conventions as the reading direction and selections, except with the edit cursor.

I don’t know how to feel about this. The arrow keys are very explicit; they are literally arrows pointing in a direction in physical space. If you were pointing at the Arabic text with your fingers and I asked you to move your finger to the right – there shouldn’t be any confusion on what direction your finger would go. If I press a key with an arrow pointing to the right, should there be any confusion on what direction to cursor moves?

What’s likely going on is the right arrow key isn’t telling the software “advance in the right direction,” but instead instructing the software to “advance forward” through the text. Or, based on what we’ve seen with the byte format, it could just be incrementing through individual characters how they’re stored in memory (logical order). So for RTL text, that means visually advancing left.

Character Directions

Some characters change based on directionality. For example, try typing an opening parenthesis character in the English portion, and also in the Arabic portion.

Reorganizing Divided Text

This one may be hard to explain, but here goes…
If I insert LTR text in the middle of RTL text, it will divide the RTL section into two segments, one on the left and one on the right. But it will automatically swap actually the visual order of the divided RTL segments.

For example, if we take our sample string, and I add some text in the middle (“INTERRUPTOR!”), look at how the RTL reorganizes (image below). Note how the Arabic text is visually different because the divided sections have been visually swapped.

Feel free to try this in the edit area above. If it’s hard to see what’s going on, the illustration below explicitly points out how the sections have been visually moved.

Why It Swaps

There are two intuitive ways to think about why this happens.

First, this is actually how the logical order is split. If we recall for RTL, the order the characters appear in memory is reverse to how they visually appear on the screen. So when we divide that with LTR text in the middle, the RTL section that ended up on the left was always on the left in memory, even before the division. And vice-versa for the RTL segment that ended up on the right.

The second way to think about this is that the message stays in the same order. In LTR, when we read a sentence, there’s the earliest part of the sentence on the left, and as we move right, we get to the later portions of the sentence. Visually maintaining the right-most part of an RTL text with the left part of an LTR text keeps the earlier and later parts of each sentence grouped.

A Demonstration To Justify The Swapping

Now to demonstrate both of those points. I can’t easily demonstrate this with Arabic because of the ligatures. So we’re going to use English twice – but we’ll pretend the blue text is supposed to be in a script that’s read as RTL. The double-English will also help provide a translation when viewing the logical order.
And please forgive any punctuation issues. I’m not sure how punctuation works when ending an LTR text with a quoted RTL script that also ends in the same (but mirror-opposite) punctuation rules.

So let’s pretend we have a text file with the logical ordering of:

He said, "Make a suggestion." The other replied, "I thought about it. I want to eat chicken tonight."

Which when displayed visually, where the blue text is written as RTL:

He said, "noitseggus a ekaM." The other replied, "thginot nekcihc tae ot tnaw I .ti tuoba thguoht I."

BTW, if the pretend RTL examples word wrap, pretend they don’t.

First, notice how if I swapped the positions of the blue left and right segments, the sentence wouldn’t make sense. While the blue text should be read backward as RTL, it’s obvious the reply (on the right) should not come before the request (on the left) when reading the English narrative. Now, what if I divide the two sentences on the right with “He paused for a second and continued,”. Let’s first look at what that logical ordering would be:

He said, "Make a suggestion." The other replied, "I thought about it." He paused for a second and continued, "I want to eat chicken tonight."

And now let’s see how the RTL text would visually look:

He said, "noitseggus a ekaM." The other replied, "ti tuoba thguoht I." He paused for a second and continued, "thginot nekcihc tae ot tnaw I."

And let’s put those two RTL sentences together to compare them:

He said, “noitseggus a ekaM.” The other replied, “thginot nekcihc tae ot tnaw I .ti tuoba thguoht I.”

He said, “noitseggus a ekaM.” The other replied, “ti tuoba thguoht I.” He paused for a second and continued, “thginot nekcihc tae ot tnaw I.”

That’s all. I’ve been starring at right-to-left text for so long, English doesn’t even look correct anymore.

For more information on the subject, here’s a technical deep-dive: The twisted road through right-to-left language support