Linear Regression from Scratch in JS — First Foray into ML for Web Developers.

There’s a lot of resources out there for machine or deep learning in JS: from brain.js to tensorflow.js but it’s always to good to know what’s happening behind the scenes, even if it’s only a glimpse.

Linear regression, or at least the concept behind it, powers a vast array of many more complicated algorithms, so it would be awesome if we can build a linear regression model from scratch, in pure JS. Let’s get right to it.

For the ones in a hurry: Codepen’s at the end of the article.

The HTML

We’ll start by adding a bit of HTML to our page so we can see what’s up and use our model in real time. I’ll use Bootstrap (V5) and Chart.js just fancy things up.

Importing Bootstrap & Chart.js

The format of our page is pretty simple. One input section to get the X values (independent variable), delimited by commas; hence, the user can input numbers in the format 1, 2, 3 and we can then read them into an array. A similar input section for the Y values (dependent variable).

Finally, we’ll have a button to trigger the model development and a canvas so we can plot the results — the HTML page now will look something like this.

Not too shabby.

Getting the values from the user

The user will need to input at least 3 numeric values in both input areas. Then we can simply get them as a string and convert that into an array of numbers to do our computations on.

Get the strings and map them into an array of numeric values

Finding the slope

Linear regression is essentially finding the line of best given the x & y values — we’ll be using the Least Squares Method to do just that.

First, we’ll find the averages of the two arrays, x_values & y_values, and then use that to compute the slope (“m” in y=m*x+b) and we’ll create a “regressor” object to store the model’s details.

Or, better yet, in JS

Awesome, now we can use these values to find the slope using the formula below

Finding “m” lets us know how steep the line is

Finding the intercept

The intercept refers to the “b” in y=m*x+b and represents the value where the line intersects the y-axis. The formula for this is relatively straightforward.

b is y_mean — slope*x_average

Predicting using our model

Now that we have our linear regression formula, we can enter the x values to see what the model predicts and then use the predictions against the true y values.

Loop through X values and feed them into the model to get y_hat (predictions)

Finding the error

To know how well our model is performing, we can use a pretty handy metric called the R2 score. R2 score, r², is, in a sense, a measure of how much prediction error is eliminated when we employ least-squares regression — with a score of 1 being a perfect fit.

R2 is simply the sum of the squares of the differences of predicted and actual values upon the sum of the squares of the differences in actual values and their mean.

Where

RSS is the sum of the squares of the differences of predicted and actual values

TSS is the sum of the squares of the differences in actual values and their mean.

We can use these formulas and elegantly find the R2 score using the values we’ve found till now.

Find the RSS & TSS and R2 will be 1 — RSS/TSS

Awesome, that’s the meaty part: we have a functioning linear regression model that is ready to predict.

Displaying the results

Calculating it all is awesome, but it would be even better if we can visualize the results. We’ll use Chart.js to plot the actual and predicted values and add the equation to the user’s screen.

Add the chart & equation to the user’s screen.

The final result should look something like this

A pretty chart always makes everything look better

The Pen

Nothing’s fun without a demo; the pen’s below and I’ve broken the functions for the sake of readability.

Machine learning’s a fun thing and hope this helps someone get started!