Multiple Regression

Formulae for multiple regression are much more compact in matrix notation. Therefore, we shall start off in the next section applying such notation first to simple regression, which we considered in Chapter 1, and then to multiple regression. After that w

PDF / 2,469,790 Bytes
32 Pages / 439.37 x 666.142 pts Page_size
42 Downloads / 192 Views

DOWNLOAD

REPORT

2

Multiple Regression 2.1

Introduction

Formulae for multiple regression are much more compact in matrix notation. Therefore, we shall start off in the next section applying such notation first to simple regression, which we considered in Chapter 1, and then to multiple regression. After that we shall derive formulae for least squares estimates and present properties of these estimates. These properties will be derived under the Gauss-Markov conditions which were presented in Chapter 1 and are essentially restated in Section 2.5.

2.2

Regression Model in Matrix Notation

We begin this section by writing the familiar straight line case of Chapter 1 in matrix notation. Recall that the regression model then was: Y1

=

ßo

+ ß1 Xl1 + f1 (2.1)

Now if we set

then it is easy to verify that (2.1) may be written as

y=Xß+€·

(2.3)

Now let us consider the case of more than one independent variable. Suppose we have k independent variables Xl, ... ,Xk; then the regression model is

(2.4)

A. Sen et al., Regression Analysis © Springer Science+Business Media New York 1990

2.2. Regression Model in Matrix Notation

29

Letting

x

= (

1:: ~':':: :0:0:0:: :~:k ) 1

•..

Xnl

and ß = (

Xnk

~o

) ,

(2.5)

ßk

the model (2.4), called the multiple regression model, may also be written in the form (2.3). The matrix X is called a design matrix. As in simple regression, the ßo term in (2.4) is often called the constant term or the intercept. Note that the first column of X, i.e., the column of l's, corresponds to it. If for some reason we do not want to keep ßo in the model, we would delete this column. As mentioned in the last chapter, the last k elements in the ith row of X constitute the ith design point of the model and an observation Yi together with its corresponding design point constitute the ith case or data point. GPA (max=4) 3.95 3.84 3.68 3.59 3.57 3.49 3.47 3.40 3.08 Verbal SAT (SATV) 74 76 66 76 76 66 71 71 57 Math. SAT (SATM) 79 71 75 74 70 67 73 79 76 EXHIBIT 2.1: Data on Grade Point Average and SAT Scores. SOURCE: Dacey (1983).

Example 2.1 For the data presented in Exhibit 2.1, we may write a multiple regression model to predict GPA on the basis of SATV and SATM as YI

=

3.95

=

ßo

+ ßI(74) + ß2(79) + EI

Y2 = 3.84 = ßo + ßl(76) + ß2(71) + 102 .............................. yg = 3.08 = ßo

+ ßl(57) + ß2(76) + lOg.

The values of Y and X would be:

Y=

3.95 3.84 3.68 3.59 3.57 3.49 3.47 3.40 3.08

andX=

1 1 1 1 1 1 1 1 1

74 76 66 76 76 66 71 71 57

79 71 75 74 70 67 73 79 76

•

30

Chapter 2. Multiple Regression

2.3

Least Squares Estimates

We obtain the least squares estimate of ß in the multiple regression model by minimizing n

S = L(Yi - ßo - ßI X li'"

ßk X ik)2 = (y - Xß)'(y - Xß)

-

i=l

= y'y - ß'X'y - y'Xß+ß'X'Xß

= y'y -

2ß'(X'y)

(2.6)

+ ß'(X'X)ß,

sinee y' Xß, being a sealar, equals ß'(X'y). In order to minimize (2.6), we eould differentiate it with respect to each ßj and set the derivative equal to zero. Or, equivalently, we ean do it more eompactly using matrix differentiation (see A

Data Loading...