You Can't Be Neutral on a Moving Train
David Colaursso
Co-director, Suffolk's Legal Innovation & Tech Lab
This is the 4th post in my series 50 Days of LIT Prompts.
If I had a motto, it might be "context matters." Except, I'm resistant to picking a singular guiding principle because, well... context matters. 😉 Earlier in the week, we wrote a template to provide definitions for words we highlighted in the browser, but we found when kicking the tires that sometimes this feel down when dealing with terms of art or initialisms claimed by competing collections of words. What was lacking in these instances was context. So, today we'll add context from the active page to the prompt template. Before we get there, let's talk context more broadly.
This week my social media feeds have focused on two things: AI and unions. The precipitating events for this are: (1) the launch of this series on Monday; and (2) the fact that my wife is on strike. We're all likely to see these two topics near each other more and more for one simple reason. Despite what you may have heard, the real answer to people's AI fears isn't copyright. It's antitrust and labor law. We saw a preview of this with last year's writers' strike. I believe training AI shouldn't be a matter for copyright, and the liability for infringing outputs should probably fall mostly on the prompter. Whether or not five justices agree with me is another matter. Either way, AI art doesn't copy human artists the way you think it does & that queasy feeling it gives you isn't about copyright. It's antitrust and labor.
Image-generating models like Dall-E and Stable Diffusion do NOT contain copies of the art they were trained on. To see why consider: (1) models like Stable Diffusion and Dall-E were trained on billions of image-description pairs; (2) the trained models, the software capable of producing AI art have sizes on the order of half a dozen gigabytes. This means there is only something like 17 bits available per image. For reference, this sentence alone contains 192 bits if encoded in the popular ASCII character set. If one could store a high-resolution image in 17 bits, a cell phone with 32GB of storage could store 15 billion photos. That's 5 photos every second for 100 years. Whatever is stored in that model, it is not a database of images. There is simply not enough space. Consequently, the images produced by the model are not collages stitched together from existing images. They are something much much weirder.
To many it seems self-evident that all AI art is composed of copies. Additionally, something strikes them as unfair and so they reason that there must be an existing prohibition against such an arrangement. Understandably, they assume such a prohibition must lie with copyright. However, the absence of any actual copies within the models suggests things may be more complicated. It is important to properly identify the root of such objections lest we make bad law or policy based on mistaken understandings. If one's objection is based on something other than copying, they are well served by properly identifying the true source. Otherwise, they risk fighting the wrong battle.
Understanding what these AI tools are doing is of the utmost importance if we are to wield them well. Which, as you may have guessed, is an excellent transition to our micro-lesson, followed immediately by prompt work.
Artificial Neural Nets
Yesterday, we saw how if we squint, a logistic regression looks a lot like an artificial neuron. Today we ask, what happens when you connect a bunch of these neurons together like so?
It's well beyond the scope of this series to explain how it is we train these networks, adjust all those weights to get the right answers, but given what we've learned this week, you should be able to see how they work once trained. You feed in some inputs (x through xn) and at the other end you can read out answers. In the above example, the network will output three numbers. These represent three different possible answers (A, B, and C). We know these answers will show up as numbers between 0 and 1, and if we've trained things properly, the right answer will be the one with the highest value. That is if we get the answers A=0.2, B=0.7, and C=0.1, we can assume there's a 70% chance the answer is B, a 20% chance the answer is A, and a 10% chance the answer is C. The number of outputs, inputs, as well as the number of nodes in the layer between is arbitrary. In practice, folks play around with different numbers to get the best results. If you make an input for every pixel in an image and confine the output to one. You can train a neural next to tell you if something is or isn't a cat by providing a lot of pictures labeled cat or not. Here's a great video that explains the operation of neural nets from a slightly different (more mathy) perspective.
We really have come a long way from predicting snow days. As I keep noting, predicting words can't be that far off. Until then, let's build something!
We'll do our building in the LIT Prompts extension. If you aren't familiar with the extension, don't worry. We'll walk you through setting things up before we start building. If you have used the LIT Prompts extension before, skip to The Prompt Pattern (Template).
Up Next
Questions or comments? I'm on Mastodon @Colarusso@mastodon.social
Setup LIT Prompts
LIT Prompts is a browser extension built at Suffolk University Law School's Legal Innovation and Technology Lab to help folks explore the use of Large Language Models (LLMs) and prompt engineering. LLMs are sentence completion machines, and prompts are the text upon which they build. Feed an LLM a prompt, and it will return a plausible-sounding follow-up (e.g., "Four score and seven..." might return "years ago our fathers brought forth..."). LIT Prompts lets users create and save prompt templates based on data from an active browser window (e.g., selected text or the whole text of a webpage) along with text from a user. Below we'll walk through a specific example.
To get started, follow the first four minutes of the intro video or the steps outlined below. Note: The video only shows Firefox, but once you've installed the extension, the steps are the same.
Install the extension
Follow the links for your browser.
- Firefox: (1) visit the extension's add-ons page; (2) click "Add to Firefox;" and (3) grant permissions.
- Chrome: (1) visit the extension's web store page; (2) click "Add to Chrome;" and (3) review permissions / "Add extension."
If you don't have Firefox, you can download it here. Would you rather use Chrome? Download it here.
Point it at an API
Here we'll walk through how to use an LLM provided by OpenAI, but you don't have to use their offering. If you're interested in alternatives, you can find them here. You can even run your LLM locally, avoiding the need to share your prompts with a third-party. If you need an OpenAI account, you can create one here. Note: when you create a new OpenAI account you are given a limited amount of free API credits. If you created an account some time ago, however, these may have expired. If your credits have expired, you will need to enter a billing method before you can use the API. You can check the state of any credits here.
Login to OpenAI, and navigate to the API documentation.
Once you are looking at the API docs, follow the steps outlined in the image above. That is:
- Select "API keys" from the left menu
- Click "+ Create new secret key"
On LIT Prompt's Templates & Settings screen, set your API Base to https://api.openai.com/v1/chat/completions and your API Key equal to the value you got above after clicking "+ Create new secret key". You get there by clicking the Templates & Settings button in the extension's popup:
- open the extension
- click on Templates & Settings
- enter the API Base and Key (under the section OpenAI-Compatible API Integration)
Once those two bits of information (the API Base and Key) are in place, you're good to go. Now you can edit, create, and run prompt templates. Just open the LIT Prompts extension, and click one of the options. I suggest, however, that you read through the Templates and Settings screen to get oriented. You might even try out a few of the preloaded prompt templates. This will let you jump right in and get your hands dirty in the next section.
If you receive an error when trying to run a template after entering your Base and Key, and you are using OpenAI, make sure to check the state of any credits here. If you don't have any credits, you will need a billing method on file.
If you found this hard to follow, consider following along with the first four minutes of the video above. It covers the same content. It focuses on Firefox, but once you've installed the extension, the steps are the same.
The Prompt Pattern (Template)
When crafting a LIT Prompts template, we use a mix of plain language and variable placeholders. Specifically, you can use double curly brackets to encase predefined variables. If the text between the brackets matches one of our predefined variable names, that section of text will be replaced with the variable's value. Today we'll be combining two variables we've worked with before, {{innerText}} and {{highlighted}}. See the extension's documentation.
The {{innerText}} variable will be replaced by the innerText of your current page (roughly speaking the hard-coded text of a page) while the {{highlighted}} variable contains any text you have highlighted/selected in the active browser tab at the time you open the extension. We use the template just like Tuesday's, highlight a word or words; run the template, and get a definition. What's different here is that we ask for a definition that considers the context of the page you are on. As you will no doubt see when you kick the tires, this improves performance over Tuesday's template.
Here's the template text.
{{innerText}}
Use the information above to explain who/what the following word/phrase is, i.e., provide a description/definition for the word(s) as used above: {{highlighted}}
And here are the template's parameters:
- Output Type:
LLM. This choice means that we'll "run" the template through an LLM (i.e., this will ping an LLM and return a result). Alternatively, we could have chosenn "Prompt," in which case the extension would return the text of the completed template. - Model:
gpt-3.5-turbo-16k. This input specifies what model we should use when running the prompt. Available models differ based on your API provider. See e.g., OpenAI's list of models. We're back to using gpt-3.5-turbo-16k because we want to be able to read a whole webpage. - Temperature:
0. Temperature runs from 0 to 1 and specifies how "random" the answer should be. To help the reply stay in the mainstream, I chose the least "creative" option—0. - Max Tokens:
250. This number specifies how long the reply can be. Tokens are chunks of text the model uses to do its thing. They don't quite match up with words but are close. 1 token is something like 3/4 of a word. Smaller token limits run faster. - JSON:
No. This asks the model to output its answer in something called JSON. We don't need to worry about that here, hence the selection of "No." - Output To:
Screen Only. We can output the first reply from the LLM to a number of places, the screen, the clipboard... Here, we're content just to have it go to the screen. - Post-run Behavior:
FULL STOP. Like the choice of output, we can decide what to do after a template runs. To keep things simple, I went with "FULL STOP." - Hide Button:
unchecked. This determines if a button is displayed for this template in the extension's popup window. Here we left the option unchecked, but sometimes when running a chain of prompts, it can be useful to hide a button.
Working with the above template
To work with the above template, you could copy it and its parameters into LIT Prompts one by one, or you could download a single prompts file and upload it from the extension's Templates & Settings screen. This will replace your existing prompts.
You can download a prompts file (the above template and its parameters) suitable for upload by clicking this button:
Kick the Tires
It's one thing to read about something and another to put what you've learned into practice. Let's see how this template performs. Some of these may seem a little familiar, but bear with me.
-
Fancy words. Here are some fancy words. See if the template can help with definitions.
- abdicate
- abet
- antidisestablishmentarianism
- beguile
- benign
- beseech
- circuitous
- clandestine
- commensurate
- Kick the tires. Let's get self-referential. Highlight the title of this section, "Kick the Tires," and run the above template. Try "A rose by any other name," and "You Can't Be Neutral on a Moving Train." Now, try "LIT," unlike last time, at least for me, I no longer get the vernacular usage but rather an explanation that this stands for the Legal Innovation and Technology Lab. That's nice segue into initialisms.
- Alphabet Soup. If you head back over to the list of U.S. Government Acronyms things should go a lot better than last time. Of course, the answers are literlly on the page. So, maybe this isn't the best test.
- You're the expert. Find a website dealing published by a community to which you belong, and see how the template does with terms of art and slang from that community.
- Worthy of Note. On Tuesday, when attempting to define the term "note," a term of art for a common form of legal scholars authored by law students, we suggested we might need to revisit and correct the fact that the template couldn't get this right. As we observed in "Alphabet Soup," it's not fair to use the template on a page where the definition is provided. So, I suggest you open this law journal contents page and select the header for the "Notes" section. My hope is that you'll be pleasantly surprised. Despite the lack of a definition, I got the following reply, "In the context of the BYU Law Review, 'Notes' refers to a section of the journal that features shorter articles written by law students. These articles typically analyze a specific legal issue or topic in a concise manner and provide insights or arguments on the subject. Notes are often written by law students as part of their coursework or as a way to contribute to legal scholarship."
TL;DR References
ICYMI, here are blubs for a selection of works I linked to in this post. If you didn't click through above, you might want to give them a look now.
- Will A.I. Become the New McKinsey? by Ted Chiang. This article explores the potential risks and consequences of artificial intelligence (A.I.) in relation to capitalism. Chiang suggests that A.I. can be seen as a management-consulting firm, similar to McKinsey & Company, which concentrates wealth and disempowers workers. He argues that A.I. currently assists capital at the expense of labor, and questions whether there is a way for A.I. to assist workers instead of management. Chiang also discusses the need for economic policies to distribute the benefits of technology appropriately, as well as the importance of critical self-examination by those building world-shaking technologies. He concludes by emphasizing the need to question the assumption that more technology is always better and to engage in the hard work of building a better world. Summary based on a draft from our day one template.
- Wherein The Copia Institute Tells The Copyright Office There's No Place For Copyright Law In AI Training by Cathy Gellis. This article outlines a comment filed by the Copia Institute with the US Copyright Office, arguing that copyright law should not apply to AI training. The comment states that copyright law should not interfere with AI training because it would impede the public's right to consume works. They argue that AI training is an extension of the public's right to use tools, including software tools, to help them consume works. The comment also notes that AI training is not the same as copying or distributing copyrighted works, as it involves the analysis and processing of information rather than the creation of new works. They conclude that copyright law should not have a role in AI training and that AI training should be considered fair use or exempt from copyright altogether.
- Copyright Liability On LLMs Should Mostly Fall On The Prompter, Not The Service by Ira Rothken. The use of large language models (LLMs) like ChatGPT has raised questions about the bounds of fair use and the responsibilities of AI developers and users in relation to copyright law. In this article Rothken proposes the "Training and Output" (TAO) Doctrine as a way to address these issues. The TAO Doctrine suggests that if an AI LLM engine is trained using copyrighted works and the outputs generated are based on user prompts, the responsibility for any potential copyright infringement should lie with the user, not the AI system. This approach recognizes the dual-use nature of AI technologies and emphasizes the importance of user intent and inputs in determining the nature of the output and any downstream usage. The TAO Doctrine aims to strike a balance between fostering innovation and respecting copyright laws.