How to Use Multiple Linear Regression | Statistics Ep. 30

0:00 Being an activist is tough.

0:01 You've got to consider so many factors.

0:04 Take one of my friends, Fuza.

0:05 She's a tired mom, furious over the lack

0:07 of city funding for good playgrounds nearby.

0:10 She's been radicalized by Leslie Nope and Parks and Recreation,

0:13 and she's prepared to step up,

0:15 starting with assembling a campaign to pressure the city

0:17 council into including more playground funding in the annual budget.

0:21 But Fuzza also knows that most relationships in the real

0:24 world depend on more than just one factor.

0:26 To get a sense of what tactics will work best in the halls of power,

0:30 she'll need to study what has made other campaigns successful or unsuccessful.

0:34 And she'll need to look at variables she can control and also ones she can't.

0:38 I'm Sabrina Cruz and this is Study Hall.

0:41 Real world statistics.

0:42 [Music] The real world is super complex,

0:52 made up of intricate relationships and dynamics.

0:55 When relationships involve two or more factors, we call them multivariate.

0:59 You'll see multivariate data everywhere.

1:01 And a really big part of statistics is

1:03 picking apart how those variables relate to each other.

1:06 We've talked before about linear models,

1:08 which show relationships between two things using a line.

1:11 Behind that line is regression,

1:13 which helps us understand relationships between two quantitative variables.

1:17 But linear models are also great when we've got multiple variables,

1:20 which is good news for playground champion

1:22 Fuzza and her unfolding city council pressure campaign.

1:26 Remember, she wants to see what works and what doesn't,

1:29 while also getting a sense of what she can and can't control.

1:32 She starts by looking at how residentled

1:34 playground campaigns in other cities have gone.

1:36 Fuzza sees that some campaigns only managed to get an extra

1:39 100th of a percent of their city budget for playgrounds, barely a few swings.

1:44 Other campaigns though secured a half a percent

1:46 over 5 years in just a few weeks.

1:48 Enough for jungle gyms, slides, and merrygorounds.

1:51 And a splash pad.

1:54 Oh my god, a splash pad.

1:56 Fuzza also knows there's plenty of factors that influence campaigning success,

2:00 like the size of the city's population

2:02 or things like parental donations to political campaigns.

2:06 And our budding Leslie nope can use that knowledge

2:08 to predict what kinds of outcomes are realistic in her city,

2:11 which will help her push for changes she might actually be able to pull off.

2:15 Fuzza's got a medical condition.

2:17 It's called caring too much, and it's incurable.

2:19 So now she's on to her next step, a statistical model comparing various cities,

2:24 so she can see how much of each city budget winds up

2:26 allocated to playgrounds following resident campaigns

2:29 based on lots of different factors.

2:31 That additional fraction of the budget

2:33 spent on playgrounds is her dependent variable,

2:35 a number she can predict using inputs

2:37 or independent variables into a statistical model.

2:40 Using multiple regression means putting more than one

2:43 input into the regression to predict the outcome,

2:45 which in this case are the data Fuzza gathered about previous campaigns.

2:49 Now, linear models really boil down to equations.

2:52 As we've seen before,

2:53 these models stem from this equation where the y is the outcome we want

2:56 to predict and the x is the input variable or the thing on the horizontal axis.

3:01 The a is the y intercept representing the predicted value

3:04 of the dependent variable when the independent variable is zero and the b

3:09 is the slope or the rate of change in the dependent

3:11 variable for a one unitit change in the independent variable.

3:15 Fusa wants to include lots of variables that influence a campaign outcome.

3:19 Messy is fun.

3:20 Okay.

3:21 But all she needs to do that is add more terms that look like B* X.

3:25 Basically, as the variable X,

3:27 like number of parents who write to the mayor changes,

3:30 it'll influence the outcome Y by some amount that depends on the value of B.

3:34 And Verusa can do that with basically as many variables as she wants.

3:38 For instance, say she used just three variables,

3:41 population size of each city, proportion of existing parks without playgrounds,

3:45 and number of parents who write to the mayor with the symbols X1, X2, and X3.

3:50 In multiple linear regression, the equation looks just the same as before,

3:54 except every one of these new variables gets a b coefficient of its own,

3:58 which determines how much influence each

4:00 of these variables has on predicting the outcome.

4:03 So the whole thing is now y is equal to a, which is still a constant,

4:07 plus b1* x1+ b2* x2 and so on.

4:11 The b values can tell Fuzza how much change she can expect in her outcome

4:15 depending on the change in her input variable increasing by a value of one.

4:19 like one extra person writing to the mayor or one extra person

4:22 in the city's population assuming that none of the other variables change.

4:26 And using statistical software,

4:28 she can use regression to perform a bunch of hypothesis tests

4:31 to figure out whether each variable in the linear model improves its accuracy.

4:35 Specifically, software tests whether the coefficient

4:37 of determination or R square

4:39 is affected by leaving a specific variable out of the model.

4:43 Here R square is the proportion of the variation

4:45 in different campaign outcomes explained by her model.

4:48 that tells her how helpful it really is for making predictions and it

4:52 can also tell her how helpful each additional predictor is to the model.

4:56 Fuzza sets a significance threshold of 5% so that any p value for a coefficient

5:00 greater than 0.05 will make her skeptical

5:03 of that variable's influence on the model.

5:06 Okay, so Fuzza has gathered six independent

5:08 variables about each campaign across 87 cities.

5:11 Tragically, none of which are Ponyie Indiana.

5:14 Some of those variables are things that she can control,

5:17 like the proportion of the campaign budget spent on advertising,

5:20 and some she can't, like the median income for households in a given city.

5:24 She also has data about the additional proportion

5:26 of each city's budget given to playgrounds after each campaign,

5:29 which she lists in percentage points.

5:31 Fuza throws all of these into a linear model so she can look at the coefficients

5:35 for each variable to tell her which ones are the most important to consider.

5:38 The first six columns represent her independent variables.

5:41 all of the stuff about the cities and campaigns

5:43 that can help predict the values in her seventh column,

5:46 the additional budget allocated for playgrounds.

5:49 Now, all Fuzza has to do is start up the add-on, which pops up in a little menu

5:52 on the right from which she selects linear regression.

5:55 In the input Y range, she specifies the column with her output while putting

5:59 her independent column variables into the input X range box.

6:03 She ticks the check marks for labels and confidence intervals,

6:06 picks a nice empty spot in her spreadsheet for the output range box,

6:09 and then after clicking continue, boom, she gets a lot of information.

6:14 For starters, she can see her model gets an R square of 0.66.

6:18 That means her model explains about 2/3 of the variation in the outcome,

6:21 which is the amount of extra funding.

6:22 And if this were a model with just one input, this R square would be the most

6:26 helpful description of how well our model performs.

6:29 But when there's more than one variable in the mix,

6:31 using the R square could lead us into a trap.

6:34 Overfitting.

6:35 If we take every single variable we can think of and throw it into a model,

6:39 some of them might push the R square

6:40 up by pure chance without really explaining anything.

6:44 The more we throw in, the bigger that risk becomes.

6:46 Like if Aruza included hundreds of variables,

6:49 there's a strong chance that one of those variables

6:51 would correlate really well with the outcome by pure chance.

6:54 that would give the model a really high R square,

6:57 but it wouldn't tell her anything genuinely important.

7:00 Basically, no matter what variables we throw into a model,

7:03 the R square will always go up, which could lead to the wrong conclusions.

7:07 But just below it is a slightly smaller number, which has a value of 0.63.

7:12 That's the adjusted R square.

7:14 It reduces the R square to account for how many variables we've thrown in.

7:18 It's just what we need to keep an eye

7:20 on how well the model performs with multiple variables.

7:22 Lower down in Fuzza's results,

7:24 she also sees every variable with its corresponding coefficient values,

7:28 like the coefficient for number of people

7:30 who write to the mayor is about 0.00003.

7:35 These coefficients show Fuza how her outcome,

7:38 the additional budget money playgrounds get,

7:40 is going to change based on her input variables.

7:42 But before she digs into the individual numbers,

7:45 Fuzua inspects the p values for the coefficients.

7:48 If a coefficient's p value crosses above Fuzua's 5% threshold,

7:52 she has reason to suspect that the variable

7:54 might not be all that important for her outcome.

7:56 For example, she finds that population size with a big old

7:59 p value of 0.999 isn't statistically significant for predicting her outcome.

8:05 But she pauses before throwing it out of her model

8:07 and starting over because now Fuzza has to consider collinearity.

8:11 That's when two variables are so correlated

8:14 with each other that it can throw things off.

8:16 For instance, there's both population and number

8:18 of people who write to the mayor.

8:20 But if roughly the same proportion of people write

8:22 to the mayor in every city during a campaign,

8:24 these two numbers would be tightly correlated because they

8:27 are both reflecting information about the same numbers of people.

8:31 And here, Fuzza finds that these variables

8:33 have a correlation of 0.74 in her data.

8:36 Trying to fit both those variables at the same time

8:38 into her model might make the coefficient for people who write

8:40 to the mayor smaller while making the one for population size bigger

8:44 even if it isn't all that important or the other way around.

8:47 And these coefficients can wiggle around causing trouble.

8:50 This screws up the tidy interpretations Fuza wants.

8:53 So she gets rid of population size and creates a new

8:55 model giving her a cleaner interpretation of the variables she cares about,

8:59 which is how many people write to the mayor.

9:01 When she does that, the R square barely changes.

9:04 But the adjusted R square actually goes up,

9:07 which is a strong sign that the overall model is working better.

9:10 And with that, Fuuza is ready to dig

9:12 into the only thing better than waffles, spreadsheet data.

9:15 Of course, now here, all of the coefficients in Fuza's spreadsheet

9:18 are in the same units as the variables.

9:21 A bigger positive coefficient for a certain

9:23 variable means a bigger slope in her model,

9:25 which could mean a bigger chance of landing playground money.

9:28 But since numbers like people who write to the mayor are in the thousands,

9:32 but percentage of the budget spent on advertising only goes from 1 to 100,

9:36 it's a little tricky to compare them.

9:38 That doesn't bother Fuzza.

9:39 After all, winning is every girl's dream.

9:41 But like Leslie, she knows it's her destiny.

9:44 Fuzza will use whatever coefficient she finds

9:46 to tweak each one for a killer campaign,

9:48 so long as their P values are below her threshold.

9:51 But as you move on in stats, sometimes you'll find that it's not so

9:54 obvious what coefficients really matter for a problem.

9:57 In those situations, we can standardize the independent variables just as we do

10:00 for data in a standard normal distribution by taking each value,

10:04 subtracting the mean, and dividing it all by the standard deviation.

10:08 The coefficients from the model would all be measured on a similar yard stick

10:11 with a change of one corresponding to a single

10:14 standard deviation change in that particular variable.

10:17 We call these standardized coefficients,

10:19 and they help us compare the coefficients

10:20 for variables on very different scales.

10:23 But for Fuzza, the unstandardized coefficients are her focus.

10:27 Basically, Fuza wants to see more

10:28 of a given city's budget going towards playgrounds.

10:31 She's on the lookout for any positive coefficients

10:33 in her spreadsheet that would indicate a big positive dependent variable.

10:37 So, for every individual person who writes to the mayor,

10:40 that number is the additional percentage of the budget

10:42 Fuzza can expect to be allocated to playgrounds.

10:45 But that's just one person.

10:46 What it means is that for every thousand people who write in, Fuzza

10:50 could expect an extra 0.03% 03% of the city budget for playgrounds.

10:54 Similarly, the coefficient for percentage of the campaign

10:57 budget spent on advertising is 0.004.

11:01 That's a positive number.

11:02 So, spending more on advertising could really help her cause.

11:05 That gives her a clear plan of action.

11:07 First, she'll fund raise.

11:08 Then, she'll go hard on advertising to other neighborhood parents

11:11 and make sure that they write to the mayor in droves.

11:14 With any luck, her nearby playground will have new equipment in the near future.

11:18 She admits it.

11:19 Fuzza is often inspired by herself.

11:22 Treat yourself to understanding multiple linear regression,

11:24 whether or not you're a playground warrior.

11:26 It's exactly the kind of cool stuff data scientists

11:29 are doing as they break the world's secrets wide open.

11:31 That's because linear relationships are clutch at predicting complex

11:34 outcomes in fields ranging from electrical engineering to economics.

11:38 A lot of science is really just statistics.

11:41 And understanding things like multiple regression is how we're able

11:44 to understand so much of all the things we consider common knowledge.

11:47 like that waffles are the best food.

11:49 Ah, actually not that.

11:52 It's tuna casserole, of course.

11:53 If you're enjoying this series and are interested in taking the full

11:56 study hall real world statistics course and earning college credit from ASU,

11:59 check out gostudyhaul.com or click on the button to learn more.

12:02 And if you want to help us out, give this video a like,

12:04 smash that subscribe button, and comment your favorite parks and wreck episode.

12:07 Thanks for watching and see you next