The Dream of Universal Translation
David Colaursso
Co-director, Suffolk's Legal Innovation & Tech Lab
This is the 13th post in my series 50 Days of LIT Prompts.
Therefore it was called Babel, because there the Lord confused the language of all the earth; and from there the Lord scattered them abroad over the face of all the earth.
- Genesis 11:9 (NRSV-CE)
The variety of human language has long been cast as an impediment to communication rather than a laboratory of descriptive experiments, each providing a unique vantage point on the human condition. In fairness, however, absent translation, the insights gained from such experiments would remain inaccessible to all but a fraction of humanity. The fear that insights from the speakers of one language will crowd out those of others is a chief concern of those who look critically on the training of Large Language Models (LLMs). Given too much weight in an LLM's training, the ideas obsessed over by the speakers of one language threaten to dominate those less commonly expressed. However, given sentence completion machines (LLMs) are trained on texts which surely include many sets of translations, they also offer the promise of translation, the promise of understanding.
Today's task is simple. We will build a tool that allows us to translate selected text into English, elicit a response to said text, and translate this response back into the original language. We can reverse the story of Babble... Um, it's probably worth noting that in the story of Babbel, when people worked together to produce advanced technology (the tower) which offered them powers rivaling that of God (occupying the heavens) it kind of ticked him off, and the only reason he didn't kill everyone was because he had made a promise not to have another flood... I bet you didn't see that coming. We backed into the whole "AI will be the end of us all" by suggesting its construction is an act of great hubris worthy of punishment from God. This starts to feel a little like the talk of AI Doomers who see in its the creation the opening of Pandora's Box. So, with all of these allusions and comparisons on the loose, I want to revisit an observation made by Ted Chiang about the fear that perhaps the end will come when we ask AI to do one "innocuous" thing, and it succeeds in doing this without regard for our wellbeing, inadvertently killing us all in the process.
If there is any lesson that we should take from stories about genies granting wishes, it's that the desire to get something without effort is the real problem. Think about the story of "The Sorcerer's Apprentice," in which the apprentice casts a spell to make broomsticks carry water but is unable to make them stop. The lesson of that story is not that magic is impossible to control: at the end of the story, the sorcerer comes back and immediately fixes the mess the apprentice made. The lesson is that you can't get out of doing the hard work. The apprentice wanted to avoid his chores, and looking for a shortcut was what got him into trouble.
The sorcerer is wiser than his apprentice because he has done the work to understand the power he wields. It is my hope that the work we are doing together in this series will lead to something similar. It's easy to simpily accept the outputs of LLMs, failing to question the context in which they were created and whether their use was responsible, to do otherwise requires an understanding that I hope we're building together. That being said, you know I think we learn best when building. So, let's build something!
We'll do our building in the LIT Prompts extension. If you aren't familiar with the LIT Prompts extension, don't worry. We'll walk you through setting things up before we start building. If you have used the LIT Prompts extension before, skip to The Prompt Pattern (Template).
Up Next
Questions or comments? I'm on Mastodon @Colarusso@mastodon.social
Setup LIT Prompts
LIT Prompts is a browser extension built at Suffolk University Law School's Legal Innovation and Technology Lab to help folks explore the use of Large Language Models (LLMs) and prompt engineering. LLMs are sentence completion machines, and prompts are the text upon which they build. Feed an LLM a prompt, and it will return a plausible-sounding follow-up (e.g., "Four score and seven..." might return "years ago our fathers brought forth..."). LIT Prompts lets users create and save prompt templates based on data from an active browser window (e.g., selected text or the whole text of a webpage) along with text from a user. Below we'll walk through a specific example.
To get started, follow the first four minutes of the intro video or the steps outlined below. Note: The video only shows Firefox, but once you've installed the extension, the steps are the same.
Install the extension
Follow the links for your browser.
- Firefox: (1) visit the extension's add-ons page; (2) click "Add to Firefox;" and (3) grant permissions.
- Chrome: (1) visit the extension's web store page; (2) click "Add to Chrome;" and (3) review permissions / "Add extension."
If you don't have Firefox, you can download it here. Would you rather use Chrome? Download it here.
Point it at an API
Here we'll walk through how to use an LLM provided by OpenAI, but you don't have to use their offering. If you're interested in alternatives, you can find them here. You can even run your LLM locally, avoiding the need to share your prompts with a third-party. If you need an OpenAI account, you can create one here. Note: when you create a new OpenAI account you are given a limited amount of free API credits. If you created an account some time ago, however, these may have expired. If your credits have expired, you will need to enter a billing method before you can use the API. You can check the state of any credits here.
Login to OpenAI, and navigate to the API documentation.
Once you are looking at the API docs, follow the steps outlined in the image above. That is:
- Select "API keys" from the left menu
- Click "+ Create new secret key"
On LIT Prompt's Templates & Settings screen, set your API Base to https://api.openai.com/v1/chat/completions and your API Key equal to the value you got above after clicking "+ Create new secret key". You get there by clicking the Templates & Settings button in the extension's popup:
- open the extension
- click on Templates & Settings
- enter the API Base and Key (under the section OpenAI-Compatible API Integration)
Once those two bits of information (the API Base and Key) are in place, you're good to go. Now you can edit, create, and run prompt templates. Just open the LIT Prompts extension, and click one of the options. I suggest, however, that you read through the Templates and Settings screen to get oriented. You might even try out a few of the preloaded prompt templates. This will let you jump right in and get your hands dirty in the next section.
If you receive an error when trying to run a template after entering your Base and Key, and you are using OpenAI, make sure to check the state of any credits here. If you don't have any credits, you will need a billing method on file.
If you found this hard to follow, consider following along with the first four minutes of the video above. It covers the same content. It focuses on Firefox, but once you've installed the extension, the steps are the same.
The Prompt Pattern (Template)
When crafting a LIT Prompts template, we use a mix of plain language and variable placeholders. Specifically, you can use double curly brackets to encase predefined variables. If the text between the brackets matches one of our predefined variable names, that section of text will be replaced with the variable's value. Today we'll be using two variables we've seen before: {{highlighted}} and {{passThrough}}. See the extension's documentation.
The {{highlighted}} variable contains any text you have highlighted/selected in the active browser tab when you open the extension. To use {{passThrough}}, however, we make use of the Post-run Behavior parameter to pass data from one template to another. If you use Post-run Behavior to send one template's output to another template, the first template's output can be read by the second template via the {{passThrough}} variable. This template's first "trick" is that the Post-run Behavior is set to "display translation and prompt," which is the name of another template. This means that after this prompt is run through an LLM, it will trigger "display translation and prompt," and pass to it this template's output. Because the Output To is set to Hidden, however, the user will not see it. The template's second "trick" is to ask for two bits of information as output: (1) the language of the selected text; and (2) a translation of the selected text into English. Next, it asks that this data be structured as JSON. That is, it asks for structured JSON data by setting the JSON parameter to Yes. This is like yesterday's template. If you haven't seen JSON before, you might want to read up on it here: https://en.wikipedia.org/wiki/JSON.
Armed with the first template's output (language and translation), the second prompt is poised to accept input from the user and translate that into the language of the selected text. Remember, if the text between our brackets is not the name of a predefined variable, like {{What is your name?}}, it will trigger a prompt for your user that echo's the placeholder (e.g., a text bubble containing, "What is your name?"). After the user answers, their reply will replace this placeholder. A list of predefined variables can be found in the extension's documentation. Here we actually feed the translated text into the brackets like this: {{{{passThrough["translation"]}}}}. Meaning the English translation of the text is shown to the user as a prompt much as "What is your name?" above. This should make sense as you read through and play with the templates below. ;)
Here's the first template's title.
Translate & reply in original language
Here's the first template's text.
You are helping translate text into English. Here is the text you are to work with:
{{highlighted}}
Return a JSON object with two key-value pairs. The first key is called `language`, and its value is the language of the above text. The second key is called `translation`, and its value is the above text translated into English.
Now return the object:
And here are the first template's parameters:
- Output Type:
LLM. This choice means that we'll "run" the template through an LLM (i.e., this will ping an LLM and return a result). Alternatively, we could have chosen "Prompt," in which case the extension would return the text of the completed template. - Model:
gpt-3.5-turbo-16k. This input specifies what model we should use when running the prompt. Available models differ based on your API provider. See e.g., OpenAI's list of models. - Temperature:
0. Temperature runs from 0 to 1 and specifies how "random" the answer should be. Since we're seeking fidelity to a text, I went with the least "creative" setting—0. - Max Tokens:
250. This number specifies how long the reply can be. Tokens are chunks of text the model uses to do its thing. They don't quite match up with words but are close. 1 token is something like 3/4 of a word. Smaller token limits run faster. - JSON:
Yes. This asks the model to output its answer in something called JSON. - Output To:
Hidden. We can output the first reply from the LLM to a number of places, the screen, the clipboard... Here, we're going to hide the conetnt as it is intended for the next template, not the user to see. - Post-run Behavior:
display translation and prompt. Like the choice of output, we can decide what to do after a template runs. Here we have chosen the name of another template. So, after this template runs it will trigger that template and pass through its output. - Hide Button:
unchecked. This determines if a button is displayed for this template in the extension's popup window.
If you're familiar with JSON, you'll recognize that the two variables below are accessing the values stored in some JSON object named passThrough. Namely, the value for "language" and "translation." In this way we can very cleanly slice up the output from the prior template. Because the Hide Button checkbox is checked below, the user will not see a button for "display translation and prompt." That of course is okay, because it is being triggered by "Translate & reply in original language."
Here's the second template's title.
display translation and prompt
Here's the second template's text.
Translate the following text into {{passThrough["language"]}}. Here's the text to translate:
{{{{passThrough["translation"]}}}}
And here are the second template's parameters:
- Output Type:
LLM. This choice means that we'll "run" the template through an LLM (i.e., this will ping an LLM and return a result). Alternatively, we could have chosen "Prompt," in which case the extension would return the text of the completed template. - Model:
gpt-3.5-turbo. This input specifies what model we should use when running the prompt. Available models differ based on your API provider. See e.g., OpenAI's list of models. Here I went with turbo instead of turbo-16k like above because I really don't think your reply needs that big of a context window. - Temperature:
0. Temperature runs from 0 to 1 and specifies how "random" the answer should be. Since we're seeking fidelity to a text, I went with the least "creative" setting—0. - Max Tokens:
250. This number specifies how long the reply can be. Tokens are chunks of text the model uses to do its thing. They don't quite match up with words but are close. 1 token is something like 3/4 of a word. Smaller token limits run faster. - JSON:
No. This asks the model to output its answer in something called JSON. We don't need to worry about that here, hence the selection of "No." - Output To:
Screen + clipboard. We can output the first reply from the LLM to a number of places, the screen, the clipboard... Here, we'll send the output to the screen and the clipboard so you can paste it right into a reply. - Post-run Behavior:
FULL STOP. Like the choice of output, we can decide what to do after a template runs. To keep things simple, I went with "FULL STOP." - Hide Button:
checked. This determines if a button is displayed for this template in the extension's popup window. Here we want to hide this button as it wouldn't work properly if triggered before the prior prompt because it is counting on the passThrough variable.
Working with the above templates
To work with the above templates, you could copy it and its parameters into LIT Prompts one by one, or you could download a single prompts file and upload it from the extension's Templates & Settings screen. This will replace your existing prompts.
You can download a prompts file (the above template and its parameters) suitable for upload by clicking this button:
Kick the Tires
It's one thing to read about something and another to put what you've learned into practice. Let's see how this template performs.
- The Morning News. If you're looking for some text to try this out on, here's a sample of widely-read newspapers I found on Wikipedia in: French, Spanish, German, Ukrainian, Japanese, and Chinese. As with prior templates, we shouldn't expect these output to be perfect, we do, however, need to ask, "compared to what," and "are useful?"
- One Word. There's no reason this prompt has to translate text into English. In fact, by changing the occurrence of "English" in the first template, you can have it work in another languages. What I find really interesting is that you can do this while keeping the rest of the prompt template in English. Go ahead, give it a try.
- Summarize and Question. Now visit an article from one of the non-English news papers and run our Summarize and question template (don't change anything about the prompt). 🤯
TL;DR References
ICYMI, here are blubs for a selection of works I linked to in this post. If you didn't click through above, you might want to give them a look now.
- On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜 by Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. There's a lot of history behind this paper. It was part of a chain of events that forced Timnit Gebru to leave Google where she was the co-lead of their ethical AI team, but more than that, it's one of the foundational papers in AI ethics, not to be confused with the field of "AI safety," which we will discuss later. It discusses several risks associated with large language models, including environmental/financial costs, biased language, lack of cultural nuance, misdirection of research, and potential for misinformation. If you want to engage critically with LLMs, this paper is a must read.
- Any sufficiently transparent magic . . . by Damien Patrick Williams. The article explores the connections between religious perspectives, myth, and magic with the development of algorithms and artificial intelligence (AI). The author argues that these elements are not only lenses to understand AI but are also foundational to its development. The article highlights the need to consider social and experiential knowledge in AI research and emphasizes the importance of engaging with marginalized voices to better understand and mitigate the harms of AI. The author also draws parallels between AI and magical beings, such as djinn, suggesting that AI systems may fulfill desires as thoroughly as they would for themselves. The article critiques the terminology and hype surrounding AI, calling for a more intentional examination of the religious and magical aspects of AI. Summary based on a draft from our day one template.
- Will A.I. Become the New McKinsey? by Ted Chiang. This article explores the potential risks and consequences of artificial intelligence (A.I.) in relation to capitalism. Chiang suggests that A.I. can be seen as a management-consulting firm, similar to McKinsey & Company, which concentrates wealth and disempowers workers. He argues that A.I. currently assists capital at the expense of labor, and questions whether there is a way for A.I. to assist workers instead of management. Chiang also discusses the need for economic policies to distribute the benefits of technology appropriately, as well as the importance of critical self-examination by those building world-shaking technologies. He concludes by emphasizing the need to question the assumption that more technology is always better and to engage in the hard work of building a better world. Summary based on a draft from our day one template.