D = { ( X 1 , Y 2 ) , … , ( X N , Y N ) } D = \{(X_{1}, Y_{2}), \dots,(X_{N}, Y_{N})\} D = {( X 1 , Y 2 ) , … , ( X N , Y N )} X i X_{i} X i Y i Y_{i } Y i
Beste era batera esanda, eredua sortu nahi dugu:
y ^ = a ∗ 0 + a ∗ 1. x ∗ 1 + ⋯ + a ∗ p . x _ p \hat{y} = a*{0} + a*{1}.x*{1} + \dots + a*{p}.x\_{p} y ^ = a ∗ 0 + a ∗ 1 . x ∗ 1 + ⋯ + a ∗ p . x _ p
non p p p X X X
Artikulu honetan arazo hau hiru eszenatokitan nola konpondu ikusiko dugu:
X dimentsio bakarrekoa denean, hau da, p = 1 p=1 p = 1
X dimentsio anitzekoa denean, hau da, p > 1 p>1 p > 1
Jaitsiera desnibela erabiliz.
X X X Sortu nahi dugun eredua formakoa da:
y ^ = a ∗ 0 + a ∗ 1. x \hat{y} = a*{0} + a*{1}.x y ^ = a ∗ 0 + a ∗ 1 . x
Gogoratu erregresio linealaren helburua datuetara hobekien egokitzen den zuzena aurkitzea dela. Beste era batera esanda, datu-puntuen eta lerroaren arteko distantzia minimizatu behar dugu.
( a ∗ 0 ^ , a ∗ 1 ^ ) = argmin ( a ∗ 0 , a ∗ 1 ) ∑ ∗ i = 1 N ( y ∗ i − y ∗ i ^ ) 2 (\hat{a*{0}}, \hat{a*{1}}) = \underset{(a*{0}, a*{1})}{\operatorname{argmin}} \sum\limits*{i=1}^{N} (y*{i} - \hat{y*{i}})^2 ( a ∗ 0 ^ , a ∗ 1 ^ ) = ( a ∗ 0 , a ∗ 1 ) argmin ∑ ∗ i = 1 N ( y ∗ i − y ∗ i ^ ) 2
= argmin ( a ∗ 0 , a ∗ 1 ) ∑ ∗ i = 1 N ( y ∗ i − ( a ∗ 0 + a ∗ 1. x ∗ i ) ) 2 = \underset{(a*{0}, a*{1})}{\operatorname{argmin}} \sum\limits*{i=1}^{N} (y*{i} - (a*{0} + a*{1}.x*{i}))^2 = ( a ∗ 0 , a ∗ 1 ) argmin ∑ ∗ i = 1 N ( y ∗ i − ( a ∗ 0 + a ∗ 1 . x ∗ i ) ) 2
Jar dezagun:
L = ∑ ∗ i = 1 N ( y ∗ i − ( a ∗ 0 + a ∗ 1. x _ i ) ) 2 L = \sum\limits*{i=1}^{N} (y*{i} - (a*{0} + a*{1}.x\_{i}))^2 L = ∑ ∗ i = 1 N ( y ∗ i − ( a ∗ 0 + a ∗ 1 . x _ i ) ) 2
Minimoa aurkitzeko, honako ekuazio hauek ebatzi behar ditugu:
{ ∂ L ∂ a 0 = 0 ∂ L ∂ a 1 = 0 \begin{cases}
\frac{\partial L}{\partial a_{0}} = 0\\
\frac{\partial L}{\partial a_{1}} = 0
\end{cases} { ∂ a 0 ∂ L = 0 ∂ a 1 ∂ L = 0 { ∑ i = 1 N − 2 ( y i − ( a 0 + a 1 . x i ) ) = 0 ∑ i = 1 N − 2 x i ( y i − ( a 0 + a 1 . x i ) ) = 0 \begin{cases}
\sum\limits_{i=1}^{N} -2(y_{i} - (a_{0} + a_{1}.x_{i})) = 0\\
\sum\limits_{i=1}^{N} -2x_{i}(y_{i} - (a_{0} + a_{1}.x_{i})) = 0
\end{cases} ⎩ ⎨ ⎧ i = 1 ∑ N − 2 ( y i − ( a 0 + a 1 . x i )) = 0 i = 1 ∑ N − 2 x i ( y i − ( a 0 + a 1 . x i )) = 0 Lehenengo ekuazioa garatzen hasiko gara:
∑ i = 1 N y i − ∑ i = 1 N a 0 + ∑ i = 1 N a 1 . x i = 0 \sum\limits_{i=1}^{N} y_{i} - \sum\limits_{i=1}^{N}a_{0} + \sum\limits_{i=1}^{N} a_{1}.x_{i} = 0\\ i = 1 ∑ N y i − i = 1 ∑ N a 0 + i = 1 ∑ N a 1 . x i = 0 ∑ i = 1 N y i − N a 0 + ∑ i = 1 N a 1 . x i = 0 \sum\limits_{i=1}^{N} y_{i} - Na_{0} + \sum\limits_{i=1}^{N} a_{1}.x_{i} = 0\\ i = 1 ∑ N y i − N a 0 + i = 1 ∑ N a 1 . x i = 0 a 0 = ∑ i = 1 N y i N − ∑ i = 1 N x i N a 1 a_{0} = \frac{\sum\limits_{i=1}^{N} y_{i}}{N} - \frac{\sum\limits_{i=1}^{N} x_{i}}{N}a_{1} a 0 = N i = 1 ∑ N y i − N i = 1 ∑ N x i a 1 a 0 = Y − X a 1 a_{0} = Y - Xa_{1} a 0 = Y − X a 1 Bigarren ekuazioan ordezkatuko dugu:
∑ i = 1 N x i ( y i − Y + X a 1 − a 1 x i ) = 0 \sum\limits_{i=1}^{N} x_{i}(y_{i} - Y + Xa_{1} - a_{1}x_{i}) = 0 i = 1 ∑ N x i ( y i − Y + X a 1 − a 1 x i ) = 0 ∑ i = 1 N ( y i − Y ) + a 1 ( X − x i ) = 0 \sum\limits_{i=1}^{N} (y_{i} - Y) + a_{1}(X - x_{i}) = 0 i = 1 ∑ N ( y i − Y ) + a 1 ( X − x i ) = 0 ∑ i = 1 N ( y i − Y ) − ∑ i = 1 N a 1 ( x i − X ) = 0 \sum\limits_{i=1}^{N} (y_{i} - Y) - \sum\limits_{i=1}^{N}a_{1}(x_{i} - X) = 0 i = 1 ∑ N ( y i − Y ) − i = 1 ∑ N a 1 ( x i − X ) = 0 a 1 = ∑ i = 1 N ( y i − Y ) ∑ i = 1 N ( x i − X ) = ∑ i = 1 N ( y i − Y ) ( x i − X ) ∑ i = 1 N ( x i − X ) 2 = C O V ( X , Y ) V A R ( X ) a_{1} = \frac{\sum\limits_{i=1}^{N} (y_{i} - Y)}{\sum\limits_{i=1}^{N}(x_{i} - X)} =
\frac{\sum\limits_{i=1}^{N} (y_{i} - Y)(x_{i} - X)}{\sum\limits_{i=1}^{N}(x_{i} - X)^2} =
\frac{COV(X, Y)}{VAR(X)} a 1 = i = 1 ∑ N ( x i − X ) i = 1 ∑ N ( y i − Y ) = i = 1 ∑ N ( x i − X ) 2 i = 1 ∑ N ( y i − Y ) ( x i − X ) = V A R ( X ) CO V ( X , Y ) Berriz ordezkatzen dugu a 0 a_{0} a 0
{ a 0 = Y − X C O V ( X , Y ) V A R ( X ) a 1 = C O V ( X , Y ) V A R ( X ) \begin{cases}
a_{0} = Y - X\frac{COV(X, Y)}{VAR(X)}\\
a_{1} = \frac{COV(X, Y)}{VAR(X)}
\end{cases} { a 0 = Y − X V A R ( X ) CO V ( X , Y ) a 1 = V A R ( X ) CO V ( X , Y ) X X X Kasu honetan, X i X_{i} X i p p p bektore bat da:
X ∗ i = ( X ∗ i 1 , X ∗ i 2 , … , X ∗ i p ) X*{i} = (X*{i1},X*{i2},\dots,X*{ip}) X ∗ i = ( X ∗ i 1 , X ∗ i 2 , … , X ∗ i p )
Beraz, eredua honela idatzita dago:
y ^ = a ∗ 0 + a ∗ 1 x ∗ 1 + a ∗ 2 x ∗ 2 + ⋯ + a ∗ p x _ p \hat{y} = a*{0} + a*{1}x*{1} + a*{2}x*{2} + \dots + a*{p}x\_{p} y ^ = a ∗ 0 + a ∗ 1 x ∗ 1 + a ∗ 2 x ∗ 2 + ⋯ + a ∗ p x _ p
edo, matrize formatuan idatz daiteke:
Y ^ = X . W \hat{Y} = X.W Y ^ = X . W
non:
Y Y Y ( N , 1 ) (N, 1) ( N , 1 )
X X X ( N , p ) (N, p) ( N , p )
W W W ( p , 1 ) (p, 1) ( p , 1 ) ( w 1 , w 2 , … , w p ) (w_{1}, w_{2}, \dots, w_{p}) ( w 1 , w 2 , … , w p )
Lehenengo kasuaren antzera, kantitate hau minimizatzea dugu helburu:
W ^ = argmin W ∑ ∗ i = 1 N ( y ∗ i − y _ i ^ ) 2 \hat{W} = \underset{W}{\operatorname{argmin}} \sum\limits*{i=1}^{N} (y*{i} - \hat{y\_{i}})^2 W ^ = W argmin ∑ ∗ i = 1 N ( y ∗ i − y _ i ^ ) 2
Jar dezagun berriro:
L = ∑ ∗ i = 1 N ( y ∗ i − y _ i ^ ) 2 L = \sum\limits*{i=1}^{N} (y*{i} - \hat{y\_{i}})^2 L = ∑ ∗ i = 1 N ( y ∗ i − y _ i ^ ) 2
= ( Y − X W ) T ( Y − X W ) = (Y-XW)^{T}(Y-XW) = ( Y − X W ) T ( Y − X W ) = Y T Y − Y T X W − W T X T Y + W T X T X W = Y^TY-Y^TXW-W^TX^TY+W^TX^TXW = Y T Y − Y T X W − W T X T Y + W T X T X W = Y T Y − 2 W T X T Y + W T X T X W = Y^TY-2W^TX^TY+W^TX^TXW = Y T Y − 2 W T X T Y + W T X T X W L L L W W W Y T Y Y^TY Y T Y W W W
∂ ( − 2 W T X T Y + W T X T X W ) ∂ W = 0 \frac{\partial (-2W^TX^TY+W^TX^TXW)}{\partial W} = 0 ∂ W ∂ ( − 2 W T X T Y + W T X T X W ) = 0 − 2 X T Y + 2 X T X W ^ = 0 -2X^TY+2X^TX\hat{W} = 0 − 2 X T Y + 2 X T X W ^ = 0 W ^ = ( X T X ) − 1 X T Y \hat{W} = (X^TX)^{-1}X^TY W ^ = ( X T X ) − 1 X T Y Hona hemen gradienteen jaitsieraren algoritmoaren formulazioa:
w ∗ n + 1 = w ∗ n − l r × ∂ f ∂ w _ n w*{n+1} = w*{n} - lr \times \frac{\partial f}{\partial w\_{n}} w ∗ n + 1 = w ∗ n − l r × ∂ w _ n ∂ f
Orain egin behar duguna da a 0 a_{0} a 0 a 1 a_{1} a 1 X X X
{ a 0 ( n + 1 ) = a 0 ( n ) − l r × ∂ L ∂ a 0 a 1 ( n + 1 ) = a 1 ( n ) − l r × ∂ L ∂ a 1 \begin{cases}
a_{0}^{(n+1)} = a_{0}^{(n)} - lr \times \frac{\partial L}{\partial a_{0}}\\
a_{1}^{(n+1)} = a_{1}^{(n)} - lr \times \frac{\partial L}{\partial a_{1}}
\end{cases} { a 0 ( n + 1 ) = a 0 ( n ) − l r × ∂ a 0 ∂ L a 1 ( n + 1 ) = a 1 ( n ) − l r × ∂ a 1 ∂ L eta badakigu:
{ ∂ L ∂ a 0 = ∑ i = 1 N − 2 ( y i − ( a 0 + a 1 . x i ) ) ∂ L ∂ a 1 = ∑ i = 1 N − 2 x i ( y i − ( a 0 + a 1 . x i ) ) \begin{cases}
\frac{\partial L}{\partial a_{0}} = \sum\limits_{i=1}^{N} -2(y_{i} - (a_{0} + a_{1}.x_{i}))\\
\frac{\partial L}{\partial a_{1}} = \sum\limits_{i=1}^{N} -2x_{i}(y_{i} - (a_{0} + a_{1}.x_{i}))
\end{cases} ⎩ ⎨ ⎧ ∂ a 0 ∂ L = i = 1 ∑ N − 2 ( y i − ( a 0 + a 1 . x i )) ∂ a 1 ∂ L = i = 1 ∑ N − 2 x i ( y i − ( a 0 + a 1 . x i )) Ordezkapen bidez:
{ a 0 ( n + 1 ) = a 0 ( n ) + 2 × l r × ∑ i = 1 N ( y i − ( a 0 ( n ) + a 1 ( n ) . x i ) ) a 1 ( n + 1 ) = a 1 ( n ) + 2 × l r × ∑ i = 1 N x i ( y i − ( a 0 ( n ) + a 1 ( n ) . x i ) ) \begin{cases}
a_{0}^{(n+1)} = a_{0}^{(n)} + 2 \times lr \times \sum\limits_{i=1}^{N} (y_{i} - (a_{0}^{(n)} + a_{1}^{(n)}.x_{i}))\\
a_{1}^{(n+1)} = a_{1}^{(n)} + 2 \times lr \times \sum\limits_{i=1}^{N} x_{i}(y_{i} - (a_{0}^{(n)} + a_{1}^{(n)}.x_{i}))
\end{cases} ⎩ ⎨ ⎧ a 0 ( n + 1 ) = a 0 ( n ) + 2 × l r × i = 1 ∑ N ( y i − ( a 0 ( n ) + a 1 ( n ) . x i )) a 1 ( n + 1 ) = a 1 ( n ) + 2 × l r × i = 1 ∑ N x i ( y i − ( a 0 ( n ) + a 1 ( n ) . x i ))
Zein da parametro optimoen bektorearen formula dimentsio anitzeko erregresio linealaren kasuan:
C O V ( X , Y ) V A R ( Y ) \frac{COV(X, Y)}{VAR(Y)} V A R ( Y ) CO V ( X , Y )
C O V ( X , Y ) V A R ( X ) \frac{COV(X, Y)}{VAR(X)} V A R ( X ) CO V ( X , Y )
( X T X ) − 1 X T Y (X^TX)^{-1}X^TY ( X T X ) − 1 X T Y "zuzena"
Zergatik jartzen dugu deribatua 0ra?
Muturrekoa aurkitzeko. "zuzena"
Deribatua minimizatzeko.
Deribatuaren zati erreala soilik gordetzea.
Zein da erregresio linealaren helburua?
Puntu guztietatik pasatzen den zuzena aurkitzea.
Datuak hobekien deskribatzen dituen lerroa aurkitzeko."zuzena"
Datuak ondoen bereizten dituen lerroa aurkitzeko.
