Should the USPTO ask AI for help with Pop Culture References?

USPTO, do you know this person? latent space "photography" by Colarusso

Rachael Dickson
Visiting Assistant Professor, Suffolk University Law School

David Colaursso
Co-director, Suffolk's Legal Innovation & Tech Lab

This is the 26th post in David's series 50 Days of LIT Prompts.

Recently, a company that is decidedly not DC Comics managed to acquire a trademark registration for "Harleen Quinzel" for use with various types of clothing goods.

You may not recognize this name off the bat. However, do you perhaps know the name Harley Quinn? The famous DC Comics villain, whose origin story is closely tied to the Joker, has appeared in 3 major motion pictures in the last 8 years (Suicide Squad, Birds of Prey, and The Suicide Squad - and yes, Suicide Squad and THE Suicide Squad are actually two different films) and is appearing in yet another one later this year (Joker: Folie à Deux). "Dr. Harleen Quinzel" is her "real" name, the Bruce Wayne to her Batman, the Norman Osborne to her Green Goblin, the Charles Xavier to her Professor X.

This registration is particularly unusual because DC Comics not only created the Harley Quinn character and many many works featuring her, but owns 11 registered Harley Quinn trademarks for use with a wide variety of goods and services, including the Class 25 clothing goods Harleen Quinzel registered with.

This is a valuable character!

As a rule, the US Patent and Trademark Office (USPTO) refuses applications to register trademarks which are likely to be confused with another, previously registered trademark. Trademarks can be found to be confusingly similar in appearance, sound, or meaning. Here, not only do HARLEY QUINN and HARLEEN QUINZEL share some similarities in terms of appearance and sound, but, as they both refer to the same character, they have an almost identical meaning. So by all rights, the prior DC Comics Harley Quinn trademark registrations should have blocked the HARLEEN QUINZEL mark.

Modified version of Wait! What? from r/memes.

This illustrates a real, ongoing problem. Currently, the trademark office's examination process trusts that examiners will recognize the relevance and meaning of marks based on pop culture references in order to know whether to refuse them. A typical likelihood of confusion search of the trademark register for similar appearing or sounding marks usually won't catch marks with a similar meaning but different appearance and sound unless the Examiner knows to search for them. To say this a different way, your average likelihood of confusion search for Clark Kent would not find a prior registration for Superman, although they share almost identical meanings and identify the same character.

Just from a practical standpoint though, it's unreasonable to expect an examiner to be familiar with all pop culture references and character names in the world. Not every trademark attorney is as familiar with superheroes and supervillains as I am! So while mulling over the situation, I reached out to David to ask about ways to help examiners of any level of pop culture savviness quickly determine the relevance or meaning of a name.

And I was like, "Wait, the problem is that the examiners need access to pop culture knowledge? Here's an app I built in like five minutes." If there's a theme to be found in this series, it's that there are a number of use cases where what appears to be a weaknesses of Large Language Models (LLMs) can be turned into a strength. Sure LLMs make things up, but that's a plus if you ask them to do improv. One of the most serious issues presented by LLMs is the fact that they encode the biases found in their training data. So, if you're LLM was largely trained on public internet discourse, you'll get an answer that looks like one from the average internet user. We saw this ourselves back when we had our LLM make poems based on the time of day. ICYMI, it produced a winter poem which betrayed an assumption that we were in the northern hemisphere. This means that the wisdom and knowledge found in small, marginalized, and exclusively off-line communities is absent from their output. Answers that are tied to US popular culture by default, however, are not a problem if you are trying to understand likely confusion in the context of the average US consumer. I don't think I'd take an LLM's output as that of a "reasonable person" (yet), but as to likelihood of confusion, if a sentence completion machines suggests the most probable words to follow your proposed mark are a description of someone else's trademarked material, I think it's worth flagging for human review. By no means do I think this will catch all potentially confusing marks, but it would have been helpful in avoiding the oversight noted above.

Below I'll show you how to have the LIT Prompts extension answer the question, "Does this have any special meaning in popular culture?" I'll also show you how to turn this into a webapp like the one I sent Rachael in response to her original email.

Yes, "Harleen Quinzel" is the real name of the popular DC Comics character Harley Quinn, who is known for her complex relationship with the Joker and her unpredictable and quirky personality.

And here's what it said when I entered "David Colarusso."

No, the name/phrase "David Colarusso" does not have any significant meaning in popular culture.

Harsh, but true. Now...

Let's build something!

We'll do our building in the LIT Prompts extension. If you aren't familiar with the LIT Prompts extension, don't worry. We'll walk you through setting things up before we start building. If you have used the LIT Prompts extension before, skip to The Prompt Pattern (Template).

Up Next

• Setup LIT Prompts
• Install the extension
• Point it at an API
• The Prompt Pattern (Template)
• Kick the Tires
• Export and Share
• TL;DR References

Questions or comments? I'm on Mastodon @Colarusso@mastodon.social

Setup LIT Prompts

7 min intro video

LIT Prompts is a browser extension built at Suffolk University Law School's Legal Innovation and Technology Lab to help folks explore the use of Large Language Models (LLMs) and prompt engineering. LLMs are sentence completion machines, and prompts are the text upon which they build. Feed an LLM a prompt, and it will return a plausible-sounding follow-up (e.g., "Four score and seven..." might return "years ago our fathers brought forth..."). LIT Prompts lets users create and save prompt templates based on data from an active browser window (e.g., selected text or the whole text of a webpage) along with text from a user. Below we'll walk through a specific example.

To get started, follow the first four minutes of the intro video or the steps outlined below. Note: The video only shows Firefox, but once you've installed the extension, the steps are the same.

Install the extension

Follow the links for your browser.

• Firefox: (1) visit the extension's add-ons page; (2) click "Add to Firefox;" and (3) grant permissions.
• Chrome: (1) visit the extension's web store page; (2) click "Add to Chrome;" and (3) review permissions / "Add extension."

If you don't have Firefox, you can download it here. Would you rather use Chrome? Download it here.

Point it at an API

Here we'll walk through how to use an LLM provided by OpenAI, but you don't have to use their offering. If you're interested in alternatives, you can find them here. You can even run your LLM locally, avoiding the need to share your prompts with a third-party. If you need an OpenAI account, you can create one here. Note: when you create a new OpenAI account you are given a limited amount of free API credits. If you created an account some time ago, however, these may have expired. If your credits have expired, you will need to enter a billing method before you can use the API. You can check the state of any credits here.

Login to OpenAI, and navigate to the API documentation.

Once you are looking at the API docs, follow the steps outlined in the image above. That is:

1. Select "API keys" from the left menu
2. Click "+ Create new secret key"

---

On LIT Prompt's Templates & Settings screen, set your API Base to https://api.openai.com/v1/chat/completions and your API Key equal to the value you got above after clicking "+ Create new secret key". You get there by clicking the Templates & Settings button in the extension's popup:

1. open the extension
2. click on Templates & Settings
3. enter the API Base and Key (under the section OpenAI-Compatible API Integration)

Once those two bits of information (the API Base and Key) are in place, you're good to go. Now you can edit, create, and run prompt templates. Just open the LIT Prompts extension, and click one of the options. I suggest, however, that you read through the Templates and Settings screen to get oriented. You might even try out a few of the preloaded prompt templates. This will let you jump right in and get your hands dirty in the next section.

If you receive an error when trying to run a template after entering your Base and Key, and you are using OpenAI, make sure to check the state of any credits here. If you don't have any credits, you will need a billing method on file.

If you found this hard to follow, consider following along with the first four minutes of the video above. It covers the same content. It focuses on Firefox, but once you've installed the extension, the steps are the same.

The Prompt Pattern (Template)

Maps are models; they don't show everything. That's okay as long as you don't confuse the map for the territory.

When crafting a LIT Prompts template, we use a mix of plain language and variable placeholders. Specifically, you can use double curly brackets to encase predefined variables. If the text between the brackets matches one of our predefined variable names, that section of text will be replaced with the variable's value. Today we'll be using . See the extension's documentation.

If the text within brackets is not the name of a predefined variable, like {{What name/phrase would you like to check for pop culture relevance?}}, it will trigger a prompt for your user that echo's the placeholder (e.g., a text bubble containing, "What name/phrase would you like to check for pop culture relevance?"). After the user answers, their reply will replace this placeholder. A list of predefined variables can be found in the extension's documentation.

To use this prompt all one has to do is launch it from the extension's popup window. However, you could also consider replacing {{What name/phrase would you like to check for pop culture relevance?}} with {{highlighted}} which would remove the user question and input any highlighted text from the active browser tab. If the name/phrase in question is already on a webpage this would save typing it.

Here's the template's title.

Reference check

Here's the template's text.

Does the name/phrase "{{What name/phrase would you like to check for pop culture relevance?}}" have any significant meaning in popular culture? If yes, explain its significance, but keep the explanation super short.

And here are the template's parameters:

• Output Type: LLM. This choice means that we'll "run" the template through an LLM (i.e., this will ping an LLM and return a result). Alternatively, we could have chosen "Prompt," in which case the extension would return the text of the completed template.
• Model: gpt-4o-mini. This input specifies what model we should use when running the prompt. Available models differ based on your API provider. See e.g., OpenAI's list of models.
• Temperature: 0.7. Temperature runs from 0 to 1 and specifies how "random" the answer should be. Here I'm using 0.7 because I'm happy to have the text be a little "creative."
• Max Tokens: 250. This number specifies how long the reply can be. Tokens are chunks of text the model uses to do its thing. They don't quite match up with words but are close. 1 token is something like 3/4 of a word. Smaller token limits run faster.
• JSON: No. This asks the model to output its answer in something called JSON. We don't need to worry about that here, hence the selection of "No."
• Output To: Screen Only. We can output the first reply from the LLM to a number of places, the screen, the clipboard... Here, we're content just to have it go to the screen.
• Post-run Behavior: FULL STOP. Like the choice of output, we can decide what to do after a template runs. To keep things simple, I went with "FULL STOP."
• Hide Button: unchecked. This determines if a button is displayed for this template in the extension's popup window.

Working with the above template

To work with the above template, you could copy it and its parameters into LIT Prompts one by one, or you could download a single prompts file and upload it from the extension's Templates & Settings screen. This will replace your existing prompts.

You can download a prompts file (the above template and its parameters) suitable for upload by clicking this button:

Kick the Tires

It's one thing to read about something and another to put what you've learned into practice. Let's see how this template performs.

• How obscure if too obscure? See how obscure your reference has to be before it starts saying, "no." Remember what we know about how these tools work and how they were trained. If it's not big on the internet, it "doesn't count."
• Add Chat. Edit the prompt to make use of the CHAT behavior so you can follow up with questions.
• Automate this. If this was to be used as an automatic tool, you'd want to make the output machine readable. Try turning on JSON mode and adding some instructions to see if you can get a nice structured answer to your question.

Export and Share

After you've made the template your own and have it behaving the way you like, you can export and share it with others. This will produce an HTML file you can share. This file should work on any internet connected device. To create your file, click the Export Interactions Page button. The contents of the textarea above the button will be appended to the top of your exported file. Importantly, if you don't want to share your API key, you should temporarily remove it from your settings before exporting.

If you want to see what an exported file looks like without having to make one yourself. You can use the buttons below. View export in browser will open the file in your browser, and Download export will download a file. In either case the following custom header will be inserted into your file. It will NOT include an API key. So, you'll have to enter one when asked if you want to see things work. This information is saved in your browser. If you've provided it before, you won't be asked again. It is not shared with me. To remove this information for this site (and only this site, not individual files), you can follow the instructions found on my privacy page. Remember, when you export your own file, whether or not it contains and API key depends on if you have one defined at the time of output.

Custom header:

<h2>Is this a pop culture reference?</h2>
<p>Enter a name/phrase to see if it's a pop-culture reference.</p> 
<hr style="border: solid 0px; border-bottom: solid 1px #555;margin: 5px 0 15px 0"/>

Not sure what's up with all those greater than and less than signs? Looking for tips on how to style your HTML? Check out this general HTML tutorial.

TL;DR References

ICYMI, if you didn't click through above, you might want to give this a look now.

• On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜 by Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. There's a lot of history behind this paper. It was part of a chain of events that forced Timnit Gebru to leave Google where she was the co-lead of their ethical AI team, but more than that, it's one of the foundational papers in AI ethics, not to be confused with the field of "AI safety," which we will discuss later. It discusses several risks associated with large language models, including environmental/financial costs, biased language, lack of cultural nuance, misdirection of research, and potential for misinformation. If you want to engage critically with LLMs, this paper is a must read.