Introduction

SynchroTales is the next generation language toolkit, designed for engaging with text-based, audio, and visual media across many languages. Our goal is to increase access to foreign language learning resources, and provide tools which empower users to build rich learning experiences catered to any learning style.

While many language apps prescribe a set of lessons or a specific order in which to learn words and grammar, SynchroTales takes a different approach. Our philosophy is that while each individual learner is different, most seek to immerse themselves in materials native to the language they're learning as quickly as possible. So rather than administering a specific learning strategy, SynchroTales provides access to a suite of tools designed to facilitate understanding of foreign language media, and help users reach the flow state where real learning happens. With SynchroTales, you can enjoy content you love in any language, and choose your own learning adventure!

Features

• Read and prepare parallel texts across thousands of language pairs, with access to hundreds of built-in translations
• Audio-book integration, with recordings in dozens of languages mapped to the source material
• Lookup millions of word definitions using the built-in dictionaries, or bring your own
• Generate in-context spaced repetition flashcards based on your current vocabulary
• Personally tailored comprehensible input from works in your collection
• View foreign-language subtitles with highlights and inline definitions tailored to your level
• Track your learning progress for each language studied

SynchroTales allows users of the native applications to super-charge their learning with cutting-edge AI technology:

• Translation models for quickly aligning and generating parallel texts
• Transcription models for synchronizing text to audio, and generating texual accompaniment for audio books, music or movies

The rest of the documentation explores this functionality in greater depth.

Getting Started

When you first visit the SynchroTales app, you will be placed on the Library screen. From this screen you will be presented with a list of items, each of which is available for reading and/or listening:

Selecting one of the items in the list will take you to the Item Details screen, from which you will see general information about the item, a list of languages it is available in, whether or not audio is available in these languages, etc.

To open an item, click on the blue button labeled "Download" - this will transfer the item to your device in all of the listed languages. Once the download is complete, click on "Open in Reader" to open the item.

You will then be transferred to the Reader screen. Depending on the default language pair configured in your Settings, the item will be opened as either a singular or parallel text. At the top of the screen, the left and right language may be configured:

Once you've set your desired language pair, click or tap any token in the text to download the default dictionary provided by SynchroTales for that language pair:

After the download is complete, you may click or tap to view the definition of any available tokens in the text:

Double-click on a token in the text to play the corresponding audio from that position. After audio playback has been initiated, a button in the bottom right corner indicates the status of the playback, and may be used to pause and resume the audio as desired:

The audio's current position is indicated by the underlined text.

Reader

The Reader screen is the focal point of the SynchroTales app, where users engage with and learn from material in their target language.

Once an item is opened from the Library screen, it can be viewed in the Reader screen. The basic anatomy of the Reader screen is as follows:

• A. Toggle button for the left (reference) language
• B. Toggle button for the right (target) language
• C. Reader action menu
• D. Title of the currently open item
• E. Author of the currently open item
• F. Title of the currently open chapter
• G. A passage, in this case German singular text

Left and Right Languages

The left and right language that the text is displayed in is controlled by toggles labeled A and B. When both toggles are set to the same language, a singular text for that language is displayed in the reader. When the toggles are set to different languages, the reader will show a parallel text between the two languages, with its passages aligned.

Passage alignment between languages is specified by the document creator, and may be accomplished through a manual or automated process.

On clicking either of these language toggle buttons, a dialog is displayed listing each language available for the current item. In this example we will change the Left language to English by selecting it from the dialog and pressing confirm:

After confirmation, the structure of the displayed text changes to a parallel text between English and German:

Singular and parallel texts

Both singular and parallel texts consist of passages. Passages are the alignable units of parallel texts, which for the default items provided by SynchroTales are typically sentences (this is determined during document preparation, see Editor). Passages are in turn composed of tokens, which support two main interactions:

1. Dictionary definitions: when clicked or tapped, the corresponding dictionary entry will be displayed for the token. For singular texts, definitions are given in the same language as the token itself. For parallel texts, definitions are given in the opposing language. That is, words from the left text will be defined in terms of the right language, and vice versa. Definitions themselves may also consist of tokens which may open additional dictionary definitions.

2. Audio: double-clicking a token will play the audio from the token's mapped position, if available.

The underlined text indicates the current position of the audio.

Changing Chapters

When an item is open in the reader, the navigation menu will be populated with additional buttons to navigate between the item's chapters:

Changing the text width

A slider control at the top of each chapter allows for changing the width of the text:

Very large text widths may impact application performance, and clipping may occur at very small widths

Actions

Several actions are available from the menu accessed at the top right of the Reader screen:

The first three actions are Study Modes, which tailor the experience to specific learning goals:

• Alternate
• Flashcards
• Input

The "Lock Screen" action darkens the screen and prevents accidental interaction with the application. The screen can be unlocked by sliding the gray square along the track to the right:

The "Help" action redirects the user to this page.

Study Modes

SynchroTales provides multiple study modes for users looking to take a focused approach to language acquisition. The existing study modes and their intended use-cases are as follows:

• Alternate: The alternate mode is intended for extensive listening in cases where you would prefer not to look at the phone or computer screen. Unlike the Input mode, you will progress through the entirety of the story in a logical and comprehensible manner, which generally makes the experience more engaging.

• Flashcards: The flashcard mode is primarily used to help the software form an estimate of one's current passive vocabulary, which facilitates use of the Input mode. It also solidifies understanding of individual words and helps with retention of previously learned vocabulary.

• Input: The input mode is designed to present your target language in context but just outside your current level of understanding. It is used for improving listening and reading comprehension, and leverages the scores recorded from the Flashcard mode.

Alternate

The Alternate study mode plays each aligned group of parallel passages, first the audio associated with the configured left language, then the audio from the configured right language. Any configured Settings for each text, such as playback rate and volume, will be respected while alternating.

To use this study mode, select "Alternate" from the action menu at the top right of the reader screen:

Alternation will begin from either the start of the chapter, or from whichever passage playback was last initiated during the current session. If you want to start alternating from the middle of a chapter, first begin standard playback from that point by double-clicking on the desired position, then select Alternate from the action menu.

If at any point you pause the playback while alternating, it may be resumed from the beginning of the most recently played passage using the replay button:

Double-clicking a token to initiate standard playback, or entering one of the other study modes, will exit the alternate mode.

Lock screen

Some smart phones suspend the ability for apps to dynamically control media playback when the screen is turned off. Since it's often desirable to use the alternate mode without looking at the screen, the "lock screen" option may be used as a substitute for turning the screen off completely:

To disable the screen lock, slide the gray square along the track from left to right.

Flashcards

Flashcards in SynchroTales are presented using the SuperMemo 2 algorithm based on the user's prior flashcard scores. This is similar to many other scheduling applications, however there are a few key differences which tailor the SynchroTales experience specifically to language learners.

Most importantly, SynchroTales flashcards are always shown in the context of the item from which the flashcard session was initiated. Individual flashcards are just "slices" of the aligned text between your chosen reference and target languages, which have been selected algorithmically based on prior scores. The parameters of the algorithm are configurable from the Flashcards session prompt, so you can tailor the experience to your preferences or learning style.

Flashcard scores are stored locally on your device and can be exported from the Progress screen.

We'll first look at the most basic example of a flashcard session to illustrate the overall process, then examine the impact of each session parameter individually.

Example flashcard session

A flashcard session can be started by selecting the "Flashcards" option from the action menu at the top right of the screen.

After selecting this option, the session configuration parameters will be shown. For the sake of simplicity, we'll proceed using the default parameters in this example:

After pressing confirm, the first flashcard will be shown, and the audio from the right (target) language will be played once. Let's examine each component of the flashcard:

A.

The text outlined in red labeled "A" is the passage selected from the target language text, and the token outlined in black (in this case the spanish word "Y") is the target token for the card. This target token is the word or phrase that the user is being prompted to evaluate their understanding of. Dictionary and audio behavior are the same here as in any other text shown in the Reader: click/tap to show any available dictionary entry, and double-click/tap to play the audio.

In this example, the user is prompted to evaluate understanding of the Spanish word "Y" in the given context.

B.

The buttons labeled 0-5 are used to record understanding of the target token, with the meaning of each button as follows:

0. No understanding at all
1. Incorrect, but the correct meaning seemed familiar
2. Incorrect, but the correct meaning seemed easy to recall
3. Correct, but only recalled with serious difficulty
4. Correct, but with some hesitation
5. Perfect understanding: instant and automatic recollection

When one of these buttons is pressed, the session will progress to the next flashcard, or to completion if there are no more cards to show. The response will be saved and used to determine how soon the token in question should next be reviewed (using the aforementioned SuperMemo algorithm).

SynchroTales also uses these scores to form an estimate of your current passive vocabulary in the target language. Any tokens for which the most recent score was 4 or 5, and which are not currently due for review, are considered known, while all other tokens are considered unknown. This vocabulary estimate is used to determine what passages in a text meet the i+1 or Comprehension % criteria (explained below) for both flashcard and input study modes. The size of this vocabulary is shown in the Progress screen for any languages that the user has studied using flashcards.

Cheating in order to inflate the software's estimate of your vocabulary is usually counter-productive, as the passages selected for use in flashcard and input sessions will quickly become too difficult!

Other flashcard components

• Pressing the button labeled "Show Parallel Text," if enabled, reveals the aligned passage from the configured reference language. This can be useful to clarify understanding of the entire passage in cases where it is difficult to piece together the meaning from strictly dictionary definitions.
• The blue replay button can be used to repeat the target language audio as many times as needed.

Session parameters

Flashcard sessions can be configured in the "Start Flashcard Session" prompt to meet the user's specific learning style or strategy. This section will describe each parameter and its impact on the resulting flashcard session.

Max session size

This parameter controls the upper bound for how many flashcards make up a given session. A session may include less than this many flashcards, but it will never have more. Generally this is used to "box" flashcard sessions and prevent them from becoming too large or overwhelming. If sessions are consistently smaller than desired, other parameters can be relaxed until this bound is met.

Vocabulary padding

The vocabulary padding parameter expands the software's estimate of the user's passive vocabulary in a controlled manner. This is useful when the item is too difficult, or the source material contains many domain-specific terms and jargon which prevent useful flashcards from being selected for review.

The specifics around how exactly vocabulary is padded are as follows:

• In order of decreasing frequency (how often a particular token appears in the text), each token is checked against to user's scores to determine whether or not it is already known.
• Of those tokens which are unknown, the specified number are incorporated into the user's vocabulary for the purpose of selecting passages for the session.

Vocabulary included as padding may still appear for evaluation as flashcard target tokens.

Passage selection

Currently there are two possibilities for the passage selection:

• i+1: passages will be selected such that the only unknown vocaulary in the text is the target token. The only exception to this is when vocabulary padding is used, in which case those padded items may appear in the same selection as the target token.
• Comprehension %: passages will be selected such that the specified % of vocabulary is known (or part of the vocabulary padding). This does not include the target token, so setting the comprehension percentage to 0% will give flashcards for any unknown token regardless of what other vocabulary appears in the passage.

Target tokens are always chosen in order of decreasing frequency in the given source material unless a word list is specified.

Review strategy

The review strategy controls how many times a flashcard will be shown in the session:

• Show each card once: flashcards will be shown only once, regardless of what score is recorded for that card.
• Show until correct: after each flashcard selected for the session has been reviewed once, each flashcard for which the score was 3 or less will be shown again. This process is repeated until all cards receive a score of 4 or 5. When using this option, only the first score given for a particular card is recorded.

Ordering

The ordering parameter controls in what order the selected flashcards will be presented:

• Frequency Descending: flashcards will be presented in order of decreasing frequency for the card's target token. This means that for a given source material cards will always appear in the same order relative to eachother. This can be useful to ensure review of the most frequently used words, but care must be taken when using this option not to memorize meaning based on the ordering of the cards, since the ordering is meaningless in the context of real language use.
• Randomized: the selected flashcards will be presented in random order.

Word list

Any Word Lists added to a user's Inventory for the target language may be selected for use here. If a word list is used, passages will only be selected if the target token is part of the provided list. This can be useful in cases where tailoring one's vocabulary to a particular set or category of words is advantageous.

Input

The Input study mode presents slices of the source material similarly to Flashcards, but for Input sessions there is no evaluation and the passages are presented continuously until the session is stopped. Passage selection takes into consideration the user's flashcard scores, and input is intended to improve performance on subsequent flashcard review. In this way, the two study modes are complementary.

The Input mode will find the longest groups of passages matching the specified parameters in each chapter. The text and audio are then presented according to the session configuration, after which the session immediately advances without requiring input from the user. A short beep delineates each presented item, allowing this mode to be used for listening without looking at the screen.

Input sessions never complete automatically, and will continue cycling through the selected material until stopped by the user.

To start an input session, select the "Input" option from the action menu at the top right of the Reader screen.

The input configuration dialog will be shown, allowing customization of the input session. Press confirm to accept the configuration and start the session:

Session parameters

Input parameters are similar to those provided for Flashcards, with some key differences. Most importantly, there is no target token for input sessions, as they're intended to focus on the meaning of whole passages or groups of passages.

This section will describe each parameter and its impact on the resulting input session.

Passage selection

As with flashcards there are 2 possibilities for passage selection:

• i+1: this option is the same as for flashcards: passages will be selected so that only 1 token is unknown, as determined from the user's prior flashcard scores.
• Comprehension %: if this option is chosen, only passages for which the specified % of vocabulary is known will be selected. Unlike with flashcards, there is no target token, so setting a comprehension percent of 100% would give only passage slices for which all contained vocabulary is known. Since input sessions select the longest matching groups of passages, the lower limit to this percentage is 50%. Below this point, the selection is likely either incomprehensible or will consist of entire chapters, which defeats the purpose of the tool.

Narration

The narration parameter controls how audio for the selected passages will be presented during the session:

• Right-Left-Right: The audio for the right (target language) text will be played, followed immediately by that from the left (reference language) text. The target language audio segment will be played again for review prior to the session progressing.
• Right Only: The audio for the right (target language) text will be played, after which the session will immediately advance to the next item.

Ordering

The ordering parameter determines the order in which selected items will be presented for the session:

• Longest passage first: items are presented in order from longest to shortest. This can be useful when looking for long comprehensible segments in the target language for extensive listening.
• Story order: items are presented in the same order that they appear in the text/audio. If one already knows the story, this can provide additional contextual clues which enhance comprehension.
• Randomized: items are presented in random order. This is the default option.

Word list

As when studying Flashcards, any Word Lists added to a user's Inventory for the target language may be selected for use here. If a word list is used, items will only be selected if one or more of the words in the list is present. This can be useful when improving comprehension of a particular category or set of words is desired.

Lock screen

Some smart phones suspend the ability for apps to dynamically control media playback when the screen is turned off. Since it's often desirable to use the input mode without looking at the screen, the "lock screen" option may be used as a substitute for turning the screen off completely:

To disable the screen lock, slide the gray square along the track from left to right.

Settings

If enabled from the Settings screen, subtitles will be shown on top of any video with the corresponding text. The following additional options are available for subtitles:

Show unknown subtitle definitions

If this functionality is enabled, the definitions of any words considered unknown based on the user's Flashcard scores will be shown below the subtitle.

Highlight known subtitles

If this functionality is enabled, any words within a subtitle considered known based on a user's Flashcard scores will be highlighted in green.

Library

The library screen shows the items available to be viewed in the Reader, and allows for management of your collection.

The anatomy of the Library screen is as follows:

• A. Library actions menu
• B. Search bar
• C. Filters & Search Parameters
• D. List of available items
• E. Item-specific actions

Actions

Search and filters

Values entered into the search bar will constrain the results to those items with any partially matching titles or authors.

Clicking the filter icon will open the Search Parameters screen, from which you can narrow the results to a particular language or set of languages, change the ordering of the result set, etc.

Items

Clicking on an individual item item will open the corresponding Item Details page, which contains additional information about the item such as the languages it is available in and whether or not audio is available.

The item-specific actions dropdown will differ depending on whether you've downloaded the item locally or not. First, those actions which are available when the item has not yet been downloaded:

• View Details: this action will open the Item Details screen for the item
• Download: this action will download the .glot file from SynchroTales to your system, so that it is available for viewing

Once the item has been downloaded, the following actions are available:

• View Details: this action will open the Item Details screen for the item
• Open in Reader: this action will open the item directly in the Reader screen
• Edit: this action will open the item in the Editor
• Delete: This action removes the glotfile and any associated media from your system.

Library Items

The item details screen provides information about each item beyond what is given in the Library list view.

The title, author and a brief summary of the item are shown if available. Additionally, a blue button indicates whether or not the item in question has been downloaded locally and is available for use in the Reader.

In the case where the item has not been downloaded, a download button is shown:

Clicking the download button will, when finished, allow the item to be opened in the Reader:

In some cases, the items provided by SynchroTales may be updated to improve translation alignments, point to new audio files, change token boundaries or formatting, etc. For any items which have been downloaded previously but have changed on the server, a button will appear to update the locally stored version with the latest changes:

Below the main item card is a list of all languages that the item is available in:

Immediately below the language name is the name of the translator of the item to that language (or "Original Language," if it is not a translation). If audio is available, the narrator(s) of the item in that langage is shown below, indicated by the headphones icon.

Selecting the action menu next to an individual language reveals language-specific actions available for the item:

The "Import Audio" action can be used to provide audio for a item if not already available, or replace the existing audio.

The mapping between tokens and audio tracks is specified by the item's glot file, see Editor for more details

The "Open Singular Text" action will open the item in the chosen language, ignoring the default languages specified in Settings.

Search Parameters

The search parameters screen allows one to narrow the list of items in the Library screen to the specified criteria.

Clicking the button labeled "Title," will open a dialog to selected the sort criteria:

Clicking the arrow button will toggle the direction of the sort between ascending and descending:

Below the sort options, language-specific filters may be applied to the library. Clicking the blue button labeled "Any" reveals a dialog to select a specific language:

Once a language is selected, the library results will be filtered to only items available in the specified language. Additional options will further refine the results for the chosen language:

• Exclude machine translations: removes any results from the selection in which the specified language is labeled as a machine translation (e.g. google translate, yandex, etc)
• Must be original language: removes any results from the selection in which the specified language is a translation
• Must include audio: removes any results from the selection for which there is no audio available in the selected language

Pressing the "Clear Filter" button will remove the filter, restoring the original result set.

Additional language filters may be added by clicking the blue plus button at the bottom right of the screen:

Collection

This screen shows the items available to be viewed in the Reader, and allows for management of your collection.

The anatomy of the Collection screen is as follows:

• A. Library actions menu
• B. Search bar
• C. Filters & Search Parameters
• D. List of available items
• E. Item-specific actions

Actions

• Import glotfile: this action opens a file dialog allowing import of glotfiles (.glot) to your collection. A glotfile contains all of the text for each language, information about alignment between passages across translations, tokens & associated audio mappings, etc. These are files that someone has prepared using the Editor or another tool. At time of writing, all glotfiles available from SynchroTales can be downloaded directly through the Library and Item Details screens, so only glotfiles that have been acquired from some other source need be imported directly.

Search and filters

Values entered into the search bar will constrain the results to those items with any partially matching titles or authors.

Clicking the filter icon will open the Search Parameters screen, from which you can narrow the results to a particular language or set of languages, change the ordering of the result set, etc.

Items

Clicking on an individual item will open the corresponding Item Details page, which contains additional information about the item such as the languages it is available in and whether or not audio is available.

Item-specific actions

• View Details: this action will open the Item Details screen for the item
• Open in Reader: this action will open the item directly to the Reader screen
• Edit: this action will open the item in the Editor
• Delete: This action removes the glotfile and any associated media from your system.

Items

The item details screen provides information about each item beyond what is given in the Collection list view.

The title, author and a brief summary of the item are shown if available.

Below the main item card is a list of all languages that the item is available in:

Immediately below the language name is the name of the translator of the item to that language (or "Original Language," if it is not a translation). If audio is available, the narrator(s) of the item in that langage is shown below, indicated by the headphones icon.

Selecting the action menu next to an individual language reveals language-specific actions available for the item:

The "Import Audio" action can be used to provide audio for a item if not already available, or replace the existing audio.

The mapping between tokens and audio tracks is specified by the item's glot file, see Editor for more details

The "Open Singular Text" action will open the item in the chosen language, ignoring the default languages specified in Settings.

Search Parameters

The search parameters screen allows one to narrow the list of items in the Library screen to the specified criteria.

Clicking the button labeled "Title," will open a dialog to selected the sort criteria:

Clicking the arrow button will toggle the direction of the sort between ascending and descending:

Below the sort options, language-specific filters may be applied to the library. Clicking the blue button labeled "Any" reveals a dialog to select a specific language:

Once a language is selected, the library results will be filtered to only items available in the specified language. Additional options will further refine the results for the chosen language:

• Exclude machine translations: removes any results from the selection in which the specified language is labeled as a machine translation (e.g. google translate, yandex, etc)
• Must be original language: removes any results from the selection in which the specified language is a translation
• Must include audio: removes any results from the selection for which there is no audio available in the selected language

Pressing the "Clear Filter" button will remove the filter, restoring the original result set.

Additional language filters may be added by clicking the blue plus button at the bottom right of the screen:

Transfers

To support those who use SynchroTales across multiple devices, functionality is provided to transfer items and their associated media between devices using WebRTC. This is useful when, for example, one edits documents on a desktop computer, but does most of their reading on a mobile device. Rather than transferring each item & its associated media files individually, then importing them on the recipient device, the transfer feature can be used to send them conveniently as a batch.

Transfers are performed in a peer-to-peer manner. Data transferred will not be processed by, transferred to, or stored by any remote server operated by SynchroTales.

Send

To send a library item to another device, select the "Transfer" action from the library item action menu. After the selected item is ready to transfer, a dialog will be shown displaying a transfer code.

To sync the item to another device, enter this code into the receive dialog. Transfer will only occur while this dialog is open. it may be stopped at any time by pressing "Cancel."

Note that anyone who obtains the displayed code may receive the item while this dialog is open

Receive

To receive a library item from another device, selecth the "Receive Transfer" action from the collection action menu, then enter the code associated with the intended transfer.

Once a connection to the sender is established, select the desired languages and press confirm. After the transfer is complete, the item will be available from the collection screen.

Tools

The tools screen lists extensions collected to support your language learning journey. SynchroTales currently supports these types of items:

1. Dictionaries

Dictionaries allow you to reveal the meaning of tokens in the Reader screen by clicking or tapping on the text:

SynchroTales provides several built-in dictionaries, but you may also upload dictionaries that you acquire elsewhere. Read more about dictionaries here.

2. Word Lists

Word lists allow you to constrain the Flashcard and Input study modes to a particular set of vocabulary:

Read more about building and using word lists here

3. Translation Models

Translation models are only available in SynchroTales native applications

Translation Models enable text-tokenization, machine translation, and automatic alignment of parallel texts from the Editor.

Read more about using translation models here

4. Transcription Models

Transcription models are only available in SynchroTales native applications

Transcription Models enable machine transcription and automatic synchronization of tokens to audio in the Editor.

Read more about using transcription models here

Dictionaries

Dictionaries are what allows you to click words in the Reader and see their meaning or translation:

The Inventory screen shows all of your downloaded or imported dictionaries, with any dictionaries currently active in the Reader indicated by the green arrow. Next to the dictionary's language pair, the number of words defined by that dictionary is shown in parenthesis:

To remove a dictionary, click the close button next to the dictionary to be removed. You will be prompted to confirm prior to its deletion.

The dictionaries that SynchroTales prompts you to download when you first click on a token are intended to cover as much of the provided text as possible, however certain words may still be undefined for a number of reasons. When no definition exists in the dictionary for a particular word, rather than the definition you will see an overlay like this:

In the example, no German-English entry for the word "Gregor" exists in the dictionary (it is common for proper nouns to be absent from the provided dictionaries).

If it is important for you to have definitions available for particular words, there are a couple things you can do:

• See if you can help get the words added to the SynchroTales dictionaries
• Create a custom dictionary with the vocabulary you need

Custom Dictionaries

SynchroTales allows importing custom dictionaries which are acquired elsewhere. To import a custom dictionary, select the import button at the top right of the Dictionaries card on the Inventory screen:

You will be instructed to select a JSON file containing the dictionary information. The provided dictionary must match the format of the following example in order for the import to succeed (extra fields are ignored):

{
    "name": "My Custom Dictionary",
    "from": "es",
    "to": "en",
    "map": {
        "hola": ["hello", "hi"],
        "adiós": ["goodbye", "farewell"]
    }
}

Here is an explanation of each field:

• name: this is the name that will show up for the dictionary in the Inventory screen.
• from: ISO language code for the target language. Words to be definied should be in this language.
• to: ISO language code for the reference language. Definitions should be written in this language.
• map: an object containing string key words to be defined, and string array values for the definitions.

Only one dictionary is allowed per language pair, so make sure it includes all of the words you need! Currently, custom dictionaries are limited to containing simple lists of definitions for each word, but part-of-speech and pronunciation information for custom dictionaries will be supported in the future.

Word Lists

Word lists may be used to constrain the Flashcard and Input study modes to a particular set of vocabulary:

The Inventory screen shows all of your imported word lists. Next to the word list's name, the number of items included in that word list are shown in parenthesis:

To delete a word list, click the close button next to the word list to be removed. You will be prompted to confirm prior to its deletion.

To import a new word list, select the import button at the top right of the Word Lists card on the Inventory screen:

You will be instructed to select a JSON file containing the word list. The provided word list match the format of the following example in order for the import to succeed (extra fields are ignored):

{
    "name": "My Spanish Word List",
    "language": "es",
    "words": ["hola", "adiós"]
}

Here is an explanation of each field:

• name: this is the name that will show up for the word list in the Inventory screen.
• language: ISO language code for the target language. Words contained in the list should be in this language, and it will only be available for use in study modes where this language has been configured as the target language.
• words: A string array of the list's words.

Translation Models

For users of the native application, downloading and enabling one of the provided translation models will enable the following new actions in the Editor screen:

Tokenize

Selecting this action from any passage's action menu will use the tokenizer associated with the selected translation model to divide the text into its constituent tokens.

Translate

When editing a parallel text, selecting this action from any passage's action menu will translate the selected passage, and create a new aligned passage containing the output in the opposing language.

Auto-Align Passages

Selecting this action from the chapter title menu when viewing a parallel text will translate each passage and align them with those passages of the opposing language.

Transcription Models

For users of the native application, downloading and enabling one of the provided transcription models will enable the following new actions in the Editor screen:

Transcribe

Selecting this action from any passage's action menu will use the selected transcription model to transcribe the corresponding audio segment to a new Passage, replacing any tokens currently contained by the passage. The new passage will include tokenization and timestamps derived from the machine transcription.

This functionality is useful in situations in which no existing text accompaniment is available for the source material, which is often this is the case for TV shows, movies, etc.

Auto-Sync Tokens

Selecting this action from the chapter title's action menu will transcribe the audio for the chapter (indicated by the duration between the first and last audio position), and adjust the timestamp for each existing token to match the transcription output.

This functionality is useful in situations where text matching the audio is already available and timestamps for each token are desired. This is often the case for audiobooks.

Progress

The progress screen lists each language the user has engaged with through SynchroTales, shows the total time spent listening to that language, and displays an estimated size of the user's vocabulary in it:

The action menu at the top right of the progress screen includes options to import and export progress:

When "Export Progress" is selected, a json file containing all of the user's scores & listening times across all languages will be downloaded.

This json file may be imported using the "Import Progress" option to restore these scores and times to SynchroTales.

Imported progress will overwrite any existing progress for conflicting languages! Make sure that the scores you're importing are the ones you want, and backup your current scores so that they can be recovered.

Editor

The editor screen allows for creation and modification of glot files. Its appearance is similar to the Reader, however the functionality is quite different. This page outlines all operations available from the editor screen.

Hotkeys for many operations are available when editing from a computer or device with connected keyboard.

Document Creation

To create a new document, select "New Document" from the editor action menu:

From the dialog, select an initial language for the document to be created in, then press confirm to create the document:

Document Modification

The following aspects of a document may be modified:

• Titles & Passages
• Tokens
• Audio
• Video
• Images
• Metadata

Save & Export

Changes made in the editor are not persisted until a document is either saved or exported.

Save to collection

To save a document to your collection, press the button labeled "Save" from the editor screen action menu:

If the saved document is one provided by SynchroTales, it will be saved as a copy of the original document.

Export .glot file

To export a document as a .glot file, press the button labeled "Export .glot" from the editor screen action menu:

Export isolated language or pair

To export a .glot file containing information for only the active language language pair or a single language, press the button labeled "Export A-B (only)" from the editor screen action menu:

Titles & Passages

Titles and passages make up the body of the document for each language. They consist of:

1. Text
2. Boundaries between words of the text, referred to as Tokens
3. Alignments, which indicate sections of the document in other languages that translate a passage

The following operations are supported for title & passage modification:

Update text

To change the text of an item/translation title, chapter title, or passage, hover the desired passage or title to change and click the button labeled with a pencil icon:

The title or passage will change to a text input, where the text can be modified as desired:

To confirm the edit, click the button labeled with a checkmark icon (or press Enter). To cancel, click the button labeled with an X (or press Escape).

Create a new passage

To create a new passage, select the button labeled "Insert Passage Below" from the action menu of any existing passage or chapter title:

An empty passage will be inserted below the selected passage or title.

Remove a passage

To remove a passage, hover the passage to remove and click the button labeled with an X icon:

Passage Alignment

Beyond simply presenting the text, passages in a particular language may be aligned with passages from other languages, indicating that they are best translated by those passages. Passage alignment impacts the display of parallel text for the document, as aligned passages will always be displayed at the same vertical level.

To update the alignment of a passage between languages, first view the parallel text for the desired languages by configuring the displayed languages at the top of the editor screen:

Then, select the button labeled "Align Passage" from the passage's action menu:

All passages will be faded except for:

• Those passages which are available for alignment with the selected passage (shaded blue on hover)
• Those passages which are already aligned with the selected passage (shaded in blue, red on hover)

Clicking a passage which is available for alignment will align that passage to the one selected, while clicking a passage which is already aligned will remove the alignment for that passage.

To confirm modifications to the alignment, click the blue button labeled with a checkmark.

Tokens

Tokens represent the boundaries between words & symbols of a title or passage's text. They are used both to lookup Dictionary definitions in the Reader, and to map text in a language to its corresponding Audio.

The following operations are supported for token modification:

Selection

Clicking on a token in the text will select it. Once a token is selected, it will be highlighted in green and the selection overlay will be displayed:

The selection overlay displays:

• the token's language
• its index within the containing passage
• the containing passage's index within the chapter
• the character offset & width within the passage or title's text
• the token's part of speech, if available
• time offset and duration for the token's audio, if available

Multi-select

Multiple tokens may be selected by holding shift while clicking another token within the same passage (tokens may not be selected across multiple passages). All selected tokens are highlighted in green:

When multiple tokens are selected, the selection overlay displays:

• the tokens' language
• the selected tokens' indices within their containing passage
• the character offset and width of the selection
• the number of tokens in the selection
• time offset and duration of the selection's corresponding audio segment, if available

Creation

To create a token, simply click on any character in a passage or title's text which is not part of an existing token. A new token will be created containing the clicked character.

Deletion

To delete the selected token or tokens, click the button labeled "Delete Token(s)" from the selection overlay action menu:

Alternatively, the Delete hotkey performs the same operation.

Shift forward/backward

Shifting a token forward increases its character offset, while shifting it backward decreases the offset.

To shift the selected token or tokens, click one of the buttons labeled either "Shift Forward" or "Shift Backward" from the selection overlay action menu:

Alternatively, the left and right arrow hotkeys perform the same operations.

Note that tokens may not be shifted such that they overlap other tokens.

Resize

Resizing a token changes the number of characters that it represents.

To resize a token, click one of the buttons labeled either "Expand Token" or "Contract Token" from the selection overlay action menu:

Alternatively, the up and down arrow hotkeys perform the same operations.

Resizing is only available for singular selections, and tokens may not be resized to overlap other tokens.

Change part of speech

A token's part of speech is indicated by a Universal POS tag (upos).

To change a token's part of speech (upos), click the button labeled with its current part of speech on the selection overlay:

A dialog will be displayed allowing selection of the desired new upos:

Once the desired part of speech is selected, press the button labeled "confirm" to save.

Split passage at token

To split a passage at a particular token, press the button labeled "Split Passage" from the selection overlay action menu:

The selected token, and all subsequent tokens from the selected passage, will be moved to a new passage immediately below the original.

Audio

SynchroTales documents are most useful when each language and its Tokens are mapped to some corresponding audio. Generally this audio will be an audiobook or other recorded reading of the text. At the document level, each language may be associated with a particular audio file. Then at a more fine-grained level, each token for a language may be mapped to a time segment of the audio. These mappings drive audio functionality in the Reader screen and its Study Modes.

The following operations are supported for modifying the audio mapping of a language and its tokens within a document:

Import audio

To import an audio file for a language in the document, press the button labeled "Import ** Audio" from the title action menu:

A file dialog will be displayed for selection of an audio file. Import of the following extensions is currently supported: .ogg, .opus, .mp3, .flac, .wav, .m4a, .wma, .aac, .caf.

Note that the ability to play the chosen audio will depend on the audio file's container, encoding, and the browser/platform running SynchroTales.

Create token audio mapping

To add an audio mapping for a token, press the button labeled with a "+" on the selection overlay:

A mapping will be added with its offset equal to the end-time for the nearest preceding token audio mapping, and a duration of zero.

Remove token audio mapping

To remove the audio mapping for a token, press the button labeled "Remove Audio" from the selection overlay action menu:

Shift token audio mapping

Shifting a token's audio mapping changes its offset.

To shift an audio mapping, click one of the buttons labeled either "Shift Audio Forward" or "Shift Audio Backward" from the selection overlay audio action menu:

This operation will increase or decrease the audio mapping's offset by 100ms.

Alternatively, the ALT+right-arrow and ALT+left-arrow hotkeys perform the same operations, respectively.

Audio resizing is only available for singular selections.

Resize token audio mapping

Resizing a token's audio mapping changes its duration.

To resize an audio mapping, click one of the buttons labeled either "Expand Audio" or "Contract Audio" from the selection overlay audio action menu:

This operation will increase or decrease the audio mapping's duration by 50ms.

Alternatively, the ALT+up-arrow and ALT+down-arrow hotkeys perform the same operations, respectively.

Audio resizing is only available for singular selections.

Video

SynchroTales documents may be associated with a single corresponding video, which will be used across all languages. Since the focus of the application is written and spoken language, video is mainly intended to be used as an adjunct for text and audio content. It is important to keep in mind the following when using, and especially editing, any documents containing video:

1. The position (time) of the video will always be held in sync with that of the currently playing audio, regardless of language. This means that the audio for all of the documents' languages and its video should already be synchronized. For the time being, synchronizing video and audio streams with eachother is outside the scope of SynchroTales, but there are other free tools such as FFmpeg which can be used to do this.
2. The audio source is still those audio files associated with each language for the document, regardless of whether a video is added. Any audio streams contained within the video are simply ignored, so they should be removed or demuxed into separate audio files for import.

Import video

To import a video file for a document, press the button labeled "Import Video" from the action menu:

A file dialog will be displayed for selection of a video file. Import of the following extensions is currently supported: .mp4, .webm, .mka.

Note that the ability to play the chosen video will depend on the video's container, encoding, and the browser/platform running SynchroTales.

Images

SynchroTales supports adding language-specific cover and chapter images for documents.

Import images

To import a cover or chapter image for a language, press the button labeled "Import Image" from the item title or chapter title action menu:

A file dialog will be displayed for selection of a PNG image file.

Currently only .png images are supported. This is intended to reduce the size of the SynchroTales WASM binary, since decoders for each supported image format must be included with the executable. In the future, support for AVIF and WebP formats will likely be added.

Remove images

To remove a language's cover or chapter image, hover the image to be removed and press the button labeled with an "X":

Metadata

A document's metadata is information displayed about a document in the Library & Item Details screens. To update a document's metadata for a particular language, hover the title in the desired language and press the button labeled "Update ** Info" from the title action menu:

From the dialog, the author(s), translator(s), narrator(s) and summary may be modified as desired.

Editor Hotkeys

• CTRL+Z: undo
• CTRL+SHIFT+Z: redo
• CTRL+A: select all tokens in passage
• CTRL+C: copy selected text
• SPACE: play selection audio
• ALT+up-arrow: expand selection audio
• ALT+down-arrow: contract selection audio
• SHIFT+up-arrow: expand selection
• SHIFT+down-arrow: contract selection
• up-arrow: expand token
• down-arrow: contract token
• TAB: select next token
• SHIFT+TAB: select previous token
• ALT+right-arrow: shift token audio forward
• ALT+left-arrow: shift token audio backward
• CTRL+ALT+right-arrow: shift chapter audio forward
• CTRL+ALT+left-arrow: shift chapter audio backward
• right-arrow: shift token forward
• left-arrow: shift token backward
• ESCAPE: clear selection
• DELETE: delete selection

Settings

From the settings screen, options can be configured separately for the left and right text shown in the Reader screen.

Default Language

Clicking the language-code button at the top right of the settings card will open a dialog to set the default language for the corresponding side of the reader. When opening an item from the Library screen, this language will be set initially if available. If the specified default language is not available for the item, a language will be arbitrarily selected from those which are available.

Playback Rate

The slider labeled "Playback Rate" controls the speed at which audio will be played from the corresponding side of the reader. The playback may be set anywhere from 0.3x to 3x the original rate.

Volume

The slider labeled "Volume" may be used to adjust the level of the corresponding audio from 0-100% of its original level.

Popup Dictionary

The checkbox labeled "Enable popup dictionary" may be used to turn dictionary functionality on or off. If this option is turned off, no popup will be shown when clicking/tapping on words in the reader, regardless of whether a matching dictionary for the language pair is available from the user's Inventory.

Support

If you encounter any difficulties while using SynchroTales, we'd love to hear about them!

Please contact us at: support@synchrotales.com

Changelog

1.0.2 (2024-09-01)

Fixes:

• Allow import of additional media formats alongside glotfiles

1.0.1 (2024-09-01)

Fixes:

• Imported media asset hash issue

1.0.0 (2024-08-03)

Features:

• Subtitle overlay for videos
• Autoscroll to passage
• AI & associated editing tools (native only)

0.7.1 (2023-11-27)

Features:

• Allow changing the Reader font size
• Next chapter shortcut in Reader
• Theme selection in settings

Fixes:

• Washed-out colors on some devices

0.7.0 (2023-11-16)

Features:

• Update all dictionaries (~50,000 new definitions)
• Multi-line text inputs in the editor
• Auto-scroll to active editor text inputs
• Version control in editor (preview only, no persistence)

Fixes:

• Improve support for text entry via alternate input methods (VNI, TELEX, etc)
• Prevent scrolling on mobile while editing text
• General performance improvements

0.6.2 (2023-10-04)

Fixes:

• Remove improperly or incompletely labeled punctuation from frequency lists
• Prevent import of unintended media files from details screen (use the editor to change audio/video)
• Fix cancelation behavior for file import
• Memory optimizations and performance improvements

0.6.1 (2023-09-28)

Fixes:

• Fix issue with audio playback on previously imported glot files

0.6.0 (2023-09-26)

Features:

• Basic video support
• Button on the Reader screen to jump to the current audio position
• Allow import of media files alongside .glot in the Library
• Add hotkeys for shifting chapter audio in the Editor

Fixes:

• Clear passage alignments when exporting glot file for an isolated language pair
• Clear token selection if containing passage is removed

0.5.4 (2023-09-18)

Fixes:

• Only show singular text button for downloaded texts

0.5.3 (2023-09-15)

Features:

• Allow export of isolated language or pair from the Editor screen

Fixes:

• Address IndexedDB access race condition when opening books

0.5.1 & 0.5.2 (2023-09-13)

Features:

• Allow audio playback if available from all token overlays on Reader screen
• Add About screen with donation link

0.5.0 (2023-09-12)

Features:

• Welcome dialog for new users on the Reader screen
• Export audio files from the book details screen

0.4.3 (2023-09-10)

Fixes:

• Improve WASM initialization error handling

0.4.2 (2023-09-08)

Features:

• Adds paste button to the sync receive dialog

Fixes:

• Error handling when paste is not permitted by the browser

0.4.1 (2023-09-06)

Features:

• Expand dictionaries for added languages (AR, CS, HI, KO, SV, TR, UK, VI)
• Copy tokens missing definition

Fixes:

• Missing glyphs for author/narrator names in some cases

0.4.0 (2023-09-03)

Features:

• Sync books between devices via WebRTC
• Support Vosk & google STT transcript formats
• Add option to shift chapter audio forward/backward
• Import chapter tokens as Spacy doc json

Fixes:

• Shader validation issue on newer Android phones

0.3.1 (2023-08-26)

Fixes:

• File input on Chrome 116 for Android

0.3.0 (2023-08-23)

Features:

• Adds editing capabilities

Fixes:

• Font fallback for some language pairs

0.2.2 (2023-08-14)

Fixes:

• Inability to remove or update hosted glot files with audio-less translations
• Buttons entering hovered state when tapped on touch-screens
• Temporary blank space when scrolling up in some views

0.2.1 (2023-08-12)

Fixes:

• Fix rendering of non-tokenized sections in Arabic text
• Improve rendering of LTR spans in bidirectional text

0.2.0 (2023-08-09)

Features:

• Study mode speech synthesis fallback for books without accompanying audio
• Adds John Stuart Mill's "On Liberty" and F. Scott Fitzgerald's "The Great Gatsby" in several languages as examples of machine translation (dictionaries not yet expanded)
• Fonts added for various languages

Changes:

• Adds warning about studying machine translations
• Adds build information to the bottom of the navigation drawer

Fixes:

• Fix audio playback state desynchronization in some cases
• Basic text rendering for bidirectional languages (RTL only)

System Requirements

SynchroTales makes use of several browser features which may not be supported by certain browsers, operating systems, or devices. This document provides information on the minimum tested browser/operating-system combinations.

iOS

• Minimum version: 15
• Supported browsers: Chrome, Safari

Android

• Minimum version: not determined (<7)
• Supported browsers: Chrome, Edge, Firefox

macOS

• Minimum version: macOS Monterey (12)
• Supported browsers: Chrome, Edge, Firefox, Opera, Safari

Windows

• Minimum version: Windows 7
• Supported browsers: Chrome, Edge, Firefox

License & Copyright

All item text, translations and audio provided through synchrotales.com is part of the public domain unless stated otherwise on the Item Details screen for that work.

Individual definitions from wiktionary.org, as included in the default SynchroTales dictionaries, are provided under the Creative Commons Attribution-ShareAlike 3.0 Unported License.

The SynchroTales application is copyright 2024 SynchroTales LLC.