AI Ouroboros: Can Today's "AI" Build Yesterday's "AI?"
David Colaursso
Co-director, Suffolk's Legal Innovation & Tech Lab
This is the 6th post in my series 50 Days of AI. Previous Post | Next Post
Today if someone mentions "AI," it's a safe bet they're talking about large language models (LLMs). In the twenty teens, however, the moniker most often referenced machine learning (ML). Below we will use today's AI (LLMs) to help make some good old-fashioned AI (ML). It's long been a joke to call statistical methods like linear and logistic regression machine learning/"AI." It's funny, however, because it's true. For nearly a decade, I've used regressions as a way to introduce AI to my students. Consequently, I'm always on the lookout for better ways to teach the subject. A while back I made a website where students can create and train their own regression models—My Toy Models—and today we'll get an LLM to build a model we can upload there to play with.
It's worth noting, however, that some folks will take issue with my characterization of ML as the OG AI, prefering instead to hang that moniker on something like expert systems. After the term artificial intelligence was coined, two main camps emerged, the symbolicists and the connectionist. The first assumed that it would be necessary for people to program rules into computers if they wanted them to perform "human" tasks like identifying images. The latter reasoned it was better to have computers "discover" the rules in data. For a time, the focus was on the symbolic approach. Results, however, didn't scale and a winter fell upon the land. With the increased availability of data and computation, the connectionist regained the spotlight thanks to a bitter lesson, more data and inferred rules tend to out perform expert-drafted rules. Now folks argue about if more data is really all one needs and many assume the future lays in a combination of the two approaches, and today we'll play a little at their intersection.
We've met regression models before. Last week we explored logistic regressions in an attempt to predict snow fall, and I hope you already see how the occupy both camps. Observations allow us to "train" these models, but the features that drive them have to be selected, which brings me to a favorite paper of mine.
The Robust Beauty of Improper Linear Models in Decision Making lives rent free in my head. I think about this paper from 1979 ALL. THE. TIME!
TL;DR: Experts can make robust linear models just by picking a few salient features based on their experience. From the paper (emphasis added):
Proper linear models are those in which predictor variables are given weights in such a way that the resulting linear composite optimally predicts some criterion of interest; examples of proper linear models are standard regression analysis... proper linear models outperform clinical intuition. Improper linear models are those in which the weights of the predictor variables are obtained by some nonoptimal method; for example, they may be obtained on the basis of intuition, derived from simulating a clinical judge's predictions, or set to be equal. This article presents evidence that even such improper linear models are superior to clinical intuition when predicting a numerical criterion from numerical predictors. In fact, unit (i.e., equal) weighting is quite robust for making such predictions.
In the parlance of machine learning the TL;DR would read "feature selection is really important." You want to choose features that actually have some causal effect on the thing you're trying to predict, and clinical intuition is pretty good at this. Additionally, mathematical models tend to be a lot less noisy than clinical intuition when used on individual cases. Even if they're not perfect, their predictions bounce around a lot less. So, I got to thinking, if simply choosing the right features can get you "most of the way" to a good model, maybe LLMs could help introduce the feature selection process for those learning how to build their own models.
Getting students to think about potential features (inputs for their models) is a good place to start when building a model. The trick is to take what you know about some process and identify things you can measure that help to determine the thing you're trying to predict.
Today's template prompts the user for a question the model should answer (what they want to predict) and returns a list of features. For example, when fed "Will there be a snow day tomorrow (i.e., will they cancel school)?" it suggested the following features: predicted snowfall, temperature, and wind speed. That's not bad. When asked, "Will I get a good night's sleep?" it suggested tracking my bedtime routine (i.e., reading, meditation, watching TV, listening to music, none), exercise time, and caffeine consumption. Again, not a bad first pass.
Not only is this a good introduction for students to feature selection, but it also helps introduce the data structure used by My Toy Models. It uses something we call JSON to represent the model and its training data, and JSON is something we'll see more of soon. I'm not convinced that the template produces the perfect feature set, but it's certainly good enough to get the idea across. So, in answer to the question, "Can today's 'AI' build yesterday's 'AI?'" the answer seems to be "in part." Of course, if we want to move beyond improper models to their proper counterparts we'll need to collect data and train the models. Luckily, that's what My Toy Models is for.
If you'd like to take this tool for a spin and you haven't installed the LIT Prompts extension, you can access a standalone version of the tool here. Enter your question, download the output, and upload it to My Toy Models. When prompted to make a new model, choose "from file" and upload the output of or LIT Prompt. Now for our micro lesson.
Micro-Lesson: Word2vec (Part 1)
Word2vec is a method for turning words into numbers, long lists of numbers actually. For example, a single word like "the" can be represented by a collection of 300 numbers like those below, something we call an embedding. Today we'll talk about how we get these numbers, and later in the week we'll discuss why they're so special. You'll want to be sure you didn't miss last week's micro-lesson on artificial neural nets.
For the record, here's what a word embedding (numeric representation) for the word "the" looks like:
[-0.082835, 0.133846, 0.051245, -0.075815, -0.040481, 0.028314, 0.019539, 0.001221, 0.124486, 0.086579, 0.046799, -0.038843, -0.005031, 0.047033, 0.000289, 0.033930, -0.007488, -0.045629, -0.038843, -0.013747, -0.015912, 0.079091, 0.019071, -0.017784, 0.019656, -0.060839, -0.061775, 0.059201, 0.023868, 0.006025, 0.027729, -0.003978, -0.013572, -0.015912, 0.077219, 0.036035, -0.124486, 0.042821, 0.065051, 0.002208, -0.021177, 0.011115, -0.050075, -0.024687, 0.040013, -0.009828, -0.010413, -0.013104, 0.076751, 0.091259, -0.015561, 0.032526, 0.049373, -0.121678, 0.003042, 0.098279, 0.010120, -0.103895, -0.011700, 0.081899, -0.104831, 0.047969, -0.074411, -0.060371, -0.018369, -0.025974, 0.092195, 0.104363, 0.058031, -0.012694, 0.025389, -0.009652, 0.025623, 0.036737, 0.021879, 0.009477, 0.059669, 0.048905, 0.073007, 0.120742, 0.020592, -0.088919, -0.036035, 0.109042, -0.063179, -0.131974, -0.096407, 0.060371, -0.089387, -0.066923, -0.060371, 0.038375, -0.016263, -0.113722, -0.003671, 0.135718, -0.008248, 0.013045, -0.112318, 0.061775, 0.080495, -0.004885, 0.031356, -0.063647, 0.037205, -0.055925, 0.007020, -0.025389, -0.058031, -0.047501, 0.026793, -0.020124, 0.051479, 0.043757, 0.070667, 0.048905, -0.000669, 0.096407, 0.040481, 0.036269, -0.116530, -0.090791, -0.037907, -0.031824, 0.061307, 0.072071, -0.079559, -0.045629, 0.046097, -0.024102, -0.011992, -0.023634, -0.020943, -0.011232, 0.031824, -0.064115, -0.119806, -0.116062, -0.054287, 0.012636, 0.066455, -0.033930, -0.106702, -0.054053, -0.008658, -0.091259, 0.035567, -0.079559, 0.003013, 0.093599, -0.002091, -0.080027, 0.070667, -0.023400, 0.089855, -0.112318, 0.069263, 0.003685, 0.019656, -0.032292, -0.008014, -0.024102, 0.040715, 0.017550, 0.064583, 0.038609, -0.021411, 0.079559, -0.106702, 0.004475, -0.057329, -0.031122, 0.016029, -0.046799, 0.007020, 0.058265, 0.059201, -0.064583, -0.025389, -0.132910, -0.018135, -0.037907, -0.032994, 0.027378, 0.014917, 0.033930, 0.038141, 0.003144, 0.068327, -0.040481, 0.014976, -0.017082, 0.011056, -0.007956, -0.082835, -0.077219, -0.051479, 0.026676, -0.044225, -0.122614, 0.018837, -0.097811, 0.040715, 0.022113, 0.005791, -0.006025, 0.030654, 0.123550, 0.088451, -0.026793, 0.011232, 0.050309, 0.057563, 0.004358, -0.002720, 0.053351, 0.072539, 0.011817, -0.018369, 0.055457, 0.082367, -0.019773, 0.049373, 0.021411, -0.006084, -0.039311, -0.016731, 0.003773, 0.037907, -0.000241, 0.010764, -0.077219, 0.055691, 0.120742, -0.022347, -0.038375, -0.047267, 0.022815, -0.029016, 0.056861, -0.016263, -0.032058, -0.000373, -0.023400, -0.005704, -0.039545, 0.104363, -0.000867, 0.071135, -0.009828, 0.126358, 0.081899, -0.022230, 0.061307, 0.023049, -0.018018, 0.080027, -0.091727, 0.018720, 0.036971, -0.007605, 0.047501, -0.075815, -0.006610, 0.008892, 0.074411, -0.087047, 0.016146, -0.000386, 0.032058, -0.008833, 0.067859, 0.030888, -0.003510, 0.038141, -0.013396, 0.059903, -0.113722, 0.017433, -0.022464, -0.022464, 0.026325, 0.038141, 0.130102, 0.108106, -0.098279, -0.098279, -0.030888, -0.037205, -0.033462, -0.008482, -0.061775, 0.010530, 0.007078, -0.025389, -0.097343, 0.029601, 0.058967, 0.062243, -0.087515]
To understand where all those numbers come from let's build a neural net. We'll have an input for every word in the English language paired with an output for every word. The network would look something like the next image but with many more inputs and outputs. The number of nodes in that firt layer is up to us. Below we've picked three, but assume that the whole-language example has 300, just like the above embedding has 300 entries.
The above image should look familiar, it's last week's neural net with a few more inputs and outputs. Also, the "logistic regressions" have been replaced by circles with outputs shown on their faces. The network has been trained in the following way. If you input a particular word by changing the corresponding value to 1, the outputs should arrange themselves to tell you how likely a word is to be next to your input word. Remember, a large value is more likely. You're playing a prediction game. You're trying to predict what a adjacent word will be a given some "random" word. Above, we can see that "it" is likely to show up next to "let" and "go", while "be" will probably find itself next to "I'll" and "back". From this, I doubt our training data is representative of all English usage. It seems oddly influenced by movie dialogue. ;)
To turn a word into an embedding all we have to do is take the values from the first nodes. For example, "it" would be [0.2, 0.3, 0.8], and "be" would be [0.1, 0.5, 0.2] in our example above. Every word will have a slightly different set of numbers coming out of these nodes. In word2vec it is common to have 300 such nodes, hence the 300-number long list above.
As for what this all means, that will have to wait. Rest assured, we're one step closer to understanding how LLMs do what they do (i.e., predict the next word in a string of words). The pieces are starting to come into focus. Of course, we have glossed over a good deal, including how such networks are trained. For the interested reader, here's the paper that introduced word2vec to the world. We'll spend a few days unpacking it. So, no need to read it unless you want to. It's rather more technical than we're shooting for here. Until then, let's build something!
We'll do our building in the LIT Prompts extension. If you aren't familiar with the LIT Prompts extension, don't worry. We'll walk you through setting things up before we start building. If you have used the LIT Prompts extension before, skip to The Prompt Pattern (Template).
Up Next
Setup LIT Prompts
LIT Prompts is a browser extension built at Suffolk University Law School's Legal Innovation and Technology Lab to help folks explore the use of Large Language Models (LLMs) and prompt engineering. LLMs are sentence completion machines, and prompts are the text upon which they build. Feed an LLM a prompt, and it will return a plausible-sounding follow-up (e.g., "Four score and seven..." might return "years ago our fathers brought forth..."). LIT Prompts lets users create and save prompt templates based on data from an active browser window (e.g., selected text or the whole text of a webpage) along with text from a user. Below we'll walk through a specific example.
To get started, follow the first four minutes of the intro video or the steps outlined below. Note: The video only shows Firefox, but once you've installed the extension, the steps are the same.
Install the extension
Follow the links for your browser.
- Firefox: (1) visit the extension's add-ons page; (2) click "Add to Firefox;" and (3) grant permissions.
- Chrome: (1) visit the extension's web store page; (2) click "Add to Chrome;" and (3) review permissions / "Add extension."
If you don't have Firefox, you can download it here. Would you rather use Chrome? Download it here.
Point it at an API
Here we'll walk through how to use an LLM provided by OpenAI, but you don't have to use their offering. If you're interested in alternatives, you can find them here. You can even run your LLM locally, avoiding the need to share your prompts with a third-party. If you need an OpenAI account, you can create one here. Note: when you create a new OpenAI account you are given a limited amount of free API credits. If you created an account some time ago, however, these may have expired. If your credits have expired, you will need to enter a billing method before you can use the API. You can check the state of any credits here.
Login to OpenAI, and navigate to the API documentation.
Once you are looking at the API docs, follow the steps outlined in the image above. That is:
- Select "API keys" from the left menu
- Click "+ Create new secret key"
On LIT Prompt's Templates & Settings screen, set your API Base to https://api.openai.com/v1/chat/completions and your API Key equal to the value you got above after clicking "+ Create new secret key". You get there by clicking the Templates & Settings button in the extension's popup:
- open the extension
- click on Templates & Settings
- enter the API Base and Key (under the section OpenAI-Compatible API Integration)
Once those two bits of information (the API Base and Key) are in place, you're good to go. Now you can edit, create, and run prompt templates. Just open the LIT Prompts extension, and click one of the options. I suggest, however, that you read through the Templates and Settings screen to get oriented. You might even try out a few of the preloaded prompt templates. This will let you jump right in and get your hands dirty in the next section.
If you receive an error when trying to run a template after entering your Base and Key, and you are using OpenAI, make sure to check the state of any credits here. If you don't have any credits, you will need a billing method on file.
If you found this hard to follow, consider following along with the first four minutes of the video above. It covers the same content. It focuses on Firefox, but once you've installed the extension, the steps are the same.
The Prompt Patterns (Templates)
When crafting a LIT Prompts template, we use a mix of plain language and variable placeholders. Specifically, you can use double curly brackets to encase predefined variables. If the text between the brackets matches one of our predefined variable names, that section of text will be replaced with the variable's value. Today we'll be using {{scratch}}. Instead of simply sharing the output of our prompt, we'll be placing it into our scratch pad and downloading its contents. FYI, if you want to look at the scratch pad, it is accessible from the extension's popup window. The button is to the right of the Settings & Templates button that you have used before.
If the text within brackets is not the name of a predefined variable, like {{What question should your model answer?}}, it will trigger a prompt for your user that echo's the placeholder (e.g., a text bubble containing, "What question should your model answer?"). After the user answers, their reply will replace this placeholder. A list of predefined variables can be found in the extension's documentation.
There are two templates below. The first suggests features in the My Toy Models file format, placing them in the Scrtach Pad, and the second downloads the file.
Don't worry too much about the details of the model's special formating. In the end you'll be downloading the file and uploading it to My Toy Models.
Here's the first template's title.
Create a toy model
Here's the template's text.
You are a data scientist building a mathematical model to answer the following question:
Q: {{What question should your model answer?}}
State whether the model's answer/target is a continuous or categorical. Now, provide the shortest list of ex-ante features a group of experts would agree to include as inputs for your model formatted as a JSON objects of the following form for continuous models (i.e., linear regressions):
```model = {
"type": "continuous",
"question": "What is X plus Y?",
"description": "This sample PRETRAINED linear model shows how a regression can model addition.",
"features": [
{
"id": "x",
"name": "X",
"type": "continuous",
"units": "no units",
"lower": -100,
"upper": 100,
"mean": 0,
"integers_only": true
},
{
"id": "y",
"name": "Y",
"type": "continuous",
"units": "no units",
"lower": -100,
"upper": 100,
"mean": 0,
"integers_only": true
}
],
"interactions": [],
"training": {
"headers": [
"recorded_on",
"x",
"y",
"target",
"note"
],
"observations": []
},
"trained_on": 0,
"coefficients": {
"intercept": 0,
"x": 1,
"y": 1
}
}```
OR
```model = {
"type": "continuous",
"question": "What is X times Y?",
"description": "This sample PRETRAINED linear model shows how to use an interaction term. Here it is used to model multiplication.",
"features": [
{
"id": "x",
"name": "X",
"type": "continuous",
"units": "no units",
"lower": -10,
"upper": 10,
"mean": 0,
"integers_only": true
},
{
"id": "y",
"name": "Y",
"type": "continuous",
"units": "no units",
"lower": -10,
"upper": 10,
"mean": 0,
"integers_only": true
}
],
"interactions": [],
"training": {
"headers": [
"recorded_on",
"x",
"y",
"target",
"note"
],
"observations": []
},
"trained_on": 0,
"coefficients": {
"intercept": 0,
"x": 0,
"y": 0,
"x__x__y": 1
}
}```
And for categorical predictors (e.g., logistic regressions) the form looks like this:
```model = {
"type": "categorical",
"target": {
"possible_answers": [
"Yes",
"No"
]
},
"question": "Will I sleep well tonight?",
"description": "A person sleep prediction model. It starts with no observations and includes an interaction between a poor night's sleep and the timing of your last coffee.",
"features": [
{
"id": "last_night_s_sleep",
"name": "Last Night's Sleep",
"type": "categorical",
"categories": [
"Poor",
"Good",
"Excellent"
],
"weights": [
1,
2,
1
]
},
{
"id": "today_s_exercise",
"name": "Today's Exercise",
"description": "Minutes exercising today (walking counts).",
"type": "continuous",
"units": "Minutes",
"lower": 0,
"upper": 120,
"mean": 15,
"integers_only": true
},
{
"id": "last_coffee",
"name": "Last coffee",
"description": "How long before bed did you have your last coffee? Enter 16 if you didn't have any coffee.",
"type": "continuous",
"units": "Hours",
"lower": 0,
"upper": 16,
"mean": 10
},
{
"id": "screen_time",
"name": "Screen Time",
"description": "How long before bed did you avoid glowing screens?",
"type": "continuous",
"units": "Minutes",
"lower": 0,
"upper": 120,
"mean": 10
},
{
"id": "bedroom_temp",
"name": "Bedroom Temp",
"description": "What was the temperature when you went to bed?",
"type": "continuous",
"units": "degrees Fahrenheit",
"lower": 60,
"upper": 110,
"mean": 68
}
],
"interactions": [
{
"id": "last_night_s_sleep_poor__x__last_coffee",
"name": "Poor Last Night's Sleep × Last Coffee",
"terms": [
{
"feature": "last_night_s_sleep",
"category": "Poor"
},
"last_coffee"
]
}
],
"training": {
"headers": [
"recorded_on",
"last_night_s_sleep-poor",
"last_night_s_sleep-good",
"last_night_s_sleep-excellent",
"today_s_exercise",
"last_coffee",
"screen_time",
"bedroom_temp",
"last_night_s_sleep_poor__x__last_coffee",
"target",
"note"
],
"observations": []
},
"trained_on": 0,
"coefficients": {
"intercept": 0,
"last_night_s_sleep-poor": 0,
"last_night_s_sleep-good": 0.5,
"last_night_s_sleep-excellent": 0.5,
"today_s_exercise": 0.5,
"last_coffee": -0.5,
"screen_time": -0.5,
"bedroom_temp": -0.5,
"last_night_s_sleep_poor__x__last_coffee": 0.5
}
}```
Be sure to use appropriate units and ranges. That is, make sure any units used are those in which the feature is commonly measured and make sure that the range is large enough to capture the expected variability. Also, choose a sensible value for the mean. Always include "interactions" equal to "[]" and in "training" under "headers", always include "recorded_on", "target", and "note" along with the features you've defined above. "trained_on" should always equal "0". Under "coefficients" always include "intercept" and do your best to set the coefficients to be directionality correct (e.g., 1 for directly proportional, -1 for inversely proportional, 0 for no impact, in which case there better be an interaction term that isn't zero). You can also take your best guess at getting the relative weights right.
If the question asks for an answer other than a continuous or categorical value, respond with the output that looks like this:
```model = {"question":"the question the model is trying to answer", "type":"uncomputable"}```
And here are the template's parameters:
- Output Type:
LLM. This choice means that we'll "run" the template through an LLM (i.e., this will ping an LLM and return a result). Alternatively, we could have chosen "Prompt," in which case the extension would return the text of the completed template. - JSON:
Yes. This asks the model to output its answer in something called JSON, which is a nice machine-readable way to structure data. See https://en.wikipedia.org/wiki/JSON - Output To:
Screen + replace scratch pad. We can output the first reply from the LLM to a number of places, the screen, the clipboard... Here, I've chosen the screen and replacing the the current text of the Scrtach Pad with this output. - Post-run Behavior:
FULL STOP. Like the choice of output, we can decide what to do after a template runs. To keep things simple, I went with "FULL STOP." - Hide Button:
unchecked. This determines if a button is displayed for this template in the extension's popup window.
This next part is introduces the "SAVE TO FILE" post-run behavior. I can think of a lot of situations when it might be nice to save the contents of the scratch pad to a file. So, this is a handy template to have in your library.
Here's the second template's title.
Download MyToyModel file
Here's the template's text.
{{scratch}}And here are the template's parameters:
- Output Type:
Prompt. By choosing "Prompt" the template runs without being submitted to an LLM. It's output is just the template after slotting in variable values. - JSON:
No. This asks the model to output its answer in something called JSON. We don't need to worry about that here, hence the selection of "No." - Output To:
Screen Only. We can output the first reply from the LLM to a number of places, the screen, the clipboard... Here, we're content just to have it go to the screen. - Post-run Behavior:
SAVE TO FILE. Like the choice of output, we can decide what to do after a template runs. Here we will save the output to a file. This will trigger your browser's download feature. - Hide Button:
unchecked. This determines if a button is displayed for this template in the extension's popup window.
Working with the above templates
To work with the above templates, you could copy them and their parameters into LIT Prompts one by one, or you could download a single prompts file and upload it from the extension's Templates & Settings screen.
You can download a prompts file (the above template and its parameters) suitable for upload by clicking this button:
Kick the Tires
It's one thing to read about something and another to put what you've learned into practice. Let's see how this template performs.
- You're the expert. Use the template to construct a model for something you know pretty well. Look at the contents of the model in your scrtach pad or in the saved text file and see how it's suggested features stack up, and when they diverge, see if you can imagine an argument for the features that were included.
- Move the sliders. Don't feel like you have to wade into the JSON output of your model. Once you upload it to My Toy Models, you should be able to play around with different inputs. Go ahead give it a go.
Export and Share
After you've made the templates your own and them behaving the way you like, you can export and share them with others. This will produce an HTML file you can share. This file should work on any internet connected device. To create your file, click the Export Scratch Pad & Interactions Page button. The contents of the textarea above the button will be appended to the top of your exported file. Importantly, if you don't want to share your API key, you can leave the password field empty when prompted to enter one. Entering a password will encrypt your key and require the user to enter the password before using the exported file.
Note: the export feature will export all of your current prompts. So, you might want to download all your prompts before deleting those you don't want to export to the web app. This will let you reload them later.
If you want to see what an exported file looks like without having to make one yourself. You can use the buttons below. View export in browser will open the file in your browser, and Download export will download a file. In either case the following custom header will be inserted into your file. It will NOT include an API key. So, you'll have to enter one when asked if you want to see things work. This information is saved in your browser. If you've provided it before, you won't be asked again. It is not shared with me. To remove this information for this site (and only this site, not individual files), you can follow the instructions found on my privacy page. Remember, when you export your own file, whether or not it contains and API key depends on if you have one defined at the time of output.
Custom header:
<h2>Get Help Building a Toy Model</h2>
<p>
Use this tool to suggest features for use in a linear or logistic regression model in the form of a <a href="https://mytoymodels.org/" target="_blank">My Toy Models</a> data file. For context check out <a href="https://sadlynothavocdinosaur.com/posts/50-days-of-ai/26-27/data-science/" target="_blank">AI Ouroboros: Can Today's "AI" Build Yesterday's "AI?" Using LLMs to suggest features for linear and logistic regression models</a>. Once the model file has been made, you can save it and upload it to My Toy Models.
</p>
<hr style="border: solid 0px; border-bottom: solid 1px #555;margin: 5px 0 15px 0"/>
Not sure what's up with all those greater than and less than signs? Looking for tips on how to style your HTML? Check out this general HTML tutorial.
TL;DR References
ICYMI, here are blubs for a selection of works I linked to in this post. If you didn't click through above, you might want to give them a look now.
-
The Robust Beauty of Improper Linear Models in Decision Making by Robyn M. Dawes.
ABSTRACT: Proper linear models are those in which predictor variables are given weights in such a way that the resulting linear composite optimally predicts some criterion of interest; examples of proper linear models are standard regression analysis, discriminant function analysis, and ridge regression analysis. Research summarized in Paul Meehl's book on clinical versus statistical prediction—and a plethora of research stimulated in part by that book—all indicates that when a numerical criterion variable (e.g., graduate grade point average) is to be predicted from numerical predictor variables, proper linear models outperform clinical intuition. Improper linear models are those in which the weights of the predictor variables are obtained by some nonoptimal method; for example, they may be obtained on the basis of intuition, derived from simulating a clinical judge's predictions, or set to be equal. This article presents evidence that even such improper linear models are superior to clinical intuition when predicting a numerical criterion from numerical predictors. In fact, unit (i.e., equal) weighting is quite robust for making such predictions. The article discusses, in some detail, the application of unit weights to decide what bullet the Denver Police Department should use. Finally, the article considers commonly raised technical, psychological, and ethical resistances to using linear models to make important social decisions and presents arguments that could weaken these resistances. - Efficient Estimation of Word Representations in Vector Space by Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. This is the paper that introduced word2vec to the world. It's a technical paper. If that scares you, consider looping back and looking at it after we finish this week's micro-lessons. It might make more sense given that perspective as we'll spend the next several posts unpacking what word2vec does and why it matters.