Numpy를 활용한 수치근사법

Numpy를 활용한 수치근사법 예제¶

회귀와 분류를 구분해서 설명해보자.

In [1]:

%pylab inline

import numpy as np
import matplotlib.pyplot as plt

Populating the interactive namespace from numpy and matplotlib

sklearn 의 make_blobs 함수를 사용¶

In [2]:

from sklearn.datasets import make_blobs

In [3]:

data, label = make_blobs(n_samples=500, centers=[[0,0]])

In [4]:

plt.hlines([0],-10,10,linestyles='dotted')
plt.vlines([0],-10,10,linestyles='dotted')
plt.scatter(data[:,0], data[:,1], alpha=0.3)
plt.axis('scaled')

Out[4]:

(-11.0, 11.0, -11.0, 11.0)

In [5]:

data = data * [3,1]

plt.hlines([0],-10,10,linestyles='dotted')
plt.vlines([0],-10,10,linestyles='dotted')
plt.scatter(data[:,0], data[:,1], alpha=0.3)
plt.axis('scaled')

Out[5]:

(-11.68869011210953, 11.249198055914658, -11.0, 11.0)

np.random.randn() 사용¶

In [6]:

X = np.random.randn(500) * 3
y = X + np.random.randn(500)

plt.hlines([0],-10,10,linestyles='dotted')
plt.vlines([0],-10,10,linestyles='dotted')
plt.scatter(X, y, alpha=0.3)
plt.axis('scaled')

Out[6]:

(-11.0, 11.0, -11.318785674614812, 11.015180270219753)

In [7]:

X = X + 5
y = y + 5

plt.hlines([0],-10,10,linestyles='dotted')
plt.vlines([0],-10,10,linestyles='dotted')
plt.scatter(X, y, alpha=0.3)
plt.axis('scaled')

Out[7]:

(-11.203647282578965,
 15.276592934158252,
 -11.218595899173925,
 15.590513882652415)

In [8]:

plt.hlines([0],-10,10,linestyles='dotted')
plt.vlines([0],-10,10,linestyles='dotted')
plt.scatter(X, y, alpha=0.3, label='samples')
plt.axis('scaled')

plt.plot([-10,10],[-10,10], 'r--', label='predict line')
plt.legend()

Out[8]:

<matplotlib.legend.Legend at 0x1fa5a8c5400>

점들을 대표할 수 있는 선을 그었다. 회귀.
오차를 정의하고 오차를 어떻게 줄일것인가 하는 알고리즘을 짜는게 중요.

In [9]:

display(X.shape, y.shape)
display(X[:5], y[:5])

(500,)

(500,)

array([ 4.41422992,  5.86641576,  3.28203739, -2.46325435,  1.49463758])

array([ 3.02344866,  5.96882205,  3.67122683, -1.83331176,  1.15069205])

In [10]:

pred_y = X

Cost = ((pred_y - y)**2).sum() / len(y)
Cost

Out[10]:

1.046750963370798

In [11]:

pred_y = X * 0.99

Cost = ((pred_y - y)**2).sum() / len(y)
Cost

Out[11]:

1.0490289662403869

In [12]:

ws = np.arange(0, 2.001, 0.1) # 기울기 범위
costs = []
min_cost = np.inf
min_w = None

for w in ws:
    pred_y = X * w
    cost = ((pred_y - y)**2).sum() / len(y)
    costs.append(cost)
    
    if cost<min_cost :
        min_cost = cost
        min_w = w
        
display(min_w, min_cost)

1.0

1.046750963370798

In [13]:

plt.plot(ws, costs, 'o-')
plt.xticks(ws, rotation=90)
plt.xlabel('w', fontsize=20)
plt.ylabel('cost', fontsize=20)
plt.title('Cost Function', fontsize=20)

Out[13]:

Text(0.5,1,'Cost Function')

w: 기울기

비용함수(cost)는 기울기(w) 에 대한 이차다항식¶

$$ \hat{y_i} = w \cdot x_i $$ $$ cost = \frac{1}{N} \sum_{i=0}^{N-1} (\hat{y_i} - y_i)^2 $$ $$ = \frac{1}{N} \sum_{i=0}^{N-1} (w \cdot x_i - y_i)^2 $$ $$ = (\frac{1}{N} \sum_{i=0}^{N-1} x_i^2) \cdot w^2 - (\frac{2}{N} \sum_{i=0}^{N-1} x_i \cdot y_i) \cdot w + (\frac{1}{N} \sum_{i=0}^{N-1} y_i^2) $$