티스토리 뷰

beginner/파이썬 딥러닝 기초

XOR을 위한 텐서플로우 딥네트웍

johh 2019. 5. 7. 00:27

 

 

 

 

XOR을 위한 텐서플로우 딥네트웍

In [1]:
from PIL import Image
Image.open('XOR.png')
Out[1]:
In [2]:
import tensorflow as tf
import numpy as np
In [3]:
x_data = np.array([[0, 0], [0, 1], [1, 0], [1, 1]], dtype=np.float32)
y_data = np.array([[0],    [1],    [1],    [0]], dtype=np.float32)
 

결과값이 bianry한 0또는 1의 값을 갖기 때문에 logistic regression을 사용하면 된다.

In [ ]:
X = tf.placeholder(tf.float32)
Y = tf.placeholder(tf.float32)
W = tf.Variable(tf.random_normal([2, 1]), name='weight')
b = tf.Variable(tf.random_normal([1]), name='bias')

# Hypothesis using sigmoid: tf.div(1., 1. + tf.exp(tf.matmul(X, W)))
hypothesis = tf.sigmoid(tf.matmul(X, W) + b)

# cost/loss function
cost = -tf.reduce_mean(Y * tf.log(hypothesis) + (1 - Y) * tf.log(1 - hypothesis))
train = tf.train.GradientDescentOptimizer(learning_rate=0.1).minimize(cost)

# Accuracy computation
# True if hypothesis>0.5 else False
predicted = tf.cast(hypothesis > 0.5, dtype=tf.float32)
accuracy = tf.reduce_mean(tf.cast(tf.equal(predicted, Y), dtype=tf.float32))

# Launch graph
with tf.Session() as sess:
    # Initialize Tensorflow variables
    sess.run(tf.global_variables_initializer())
    
    for step in range(10001):
        sess.run(train, feed_dict={X: x_data, Y: y_data})
        if step % 100 == 0:
            print(step, sess.run(cost, feed_dict={X: x_data, Y: y_data}), sess.run(W))
    
    # Accuracy report
    h, c, a = sess.run([hypothesis, predicted, accuracy], feed_dict={X: x_data, Y: y_data})
    print("\nHypothesis: ", h, "\nCorrect: ", c, "\nAccuracy ", a)
In [ ]:
...

9600 0.6931472 [[1.3230493e-07]
 [1.3282292e-07]]
9700 0.6931472 [[1.3230493e-07]
 [1.3282292e-07]]
9800 0.6931472 [[1.3230493e-07]
 [1.3282292e-07]]
9900 0.6931472 [[1.3230493e-07]
 [1.3282292e-07]]
10000 0.6931472 [[1.3230493e-07]
 [1.3282292e-07]]

Hypothesis:  [[0.5]
 [0.5]
 [0.5]
 [0.5]] 
Correct:  [[0.]
 [0.]
 [0.]
 [0.]] 
Accuracy  0.5
 

결과값의 정확도가 좋지 않다!!
이론시간에 말했던 것처럼 여러단을 사용해야 한다.

 

Neural Net

In [7]:
Image.open('nn.png')
Out[7]:
In [ ]:
W = tf.Variable(tf.random_normal([2, 1]), name='weight')
b = tf.Variable(tf.random_normal([1]), name='bias')

# Hypothesis using sigmoid: tf.div(1., 1. + tf.exp(tf.matmul(X, W)))
hypothesis = tf.sigmoid(tf.matmul(X, W) + b)
 

위 그림을 구현하기 위해 수정하자

In [ ]:
W1 = tf.Variable(tf.random_normal([2, 2]), name='weight1') # weight의 크기 2x2
b1 = tf.Variable(tf.random_normal([2]), name='bias1')
layer1 = tf.sigmoid(tf.matmul(X, W1) + b1)

W2 = tf.Variable(tf.random_normal([2, 1]), name='weight2')
b2 = tf.Variable(tf.random_normal([1]), name='bias2')
hypothesis = tf.sigmoid(tf.matmul(layer1, W2) + b2)
 

식에서 layer1이 hypothesis의 입력값으로 들어가는 것을 확인 할 수 있다. 1단이 2단으로 바꼈다.

 

구현

In [ ]:
x_data = np.array([[0, 0], [0, 1], [1, 0], [1, 1]], dtype=np.float32)
y_data = np.array([[0],    [1],    [1],    [0]], dtype=np.float32)
X = tf.placeholder(tf.float32)
Y = tf.placeholder(tf.float32)

W1 = tf.Variable(tf.random_normal([2, 2]), name='weight1') # weight의 크기 2x2
b1 = tf.Variable(tf.random_normal([2]), name='bias1')
layer1 = tf.sigmoid(tf.matmul(X, W1) + b1)

W2 = tf.Variable(tf.random_normal([2, 1]), name='weight2')
b2 = tf.Variable(tf.random_normal([1]), name='bias2')
hypothesis = tf.sigmoid(tf.matmul(layer1, W2) + b2)

# cost/loss function
cost = -tf.reduce_mean(Y * tf.log(hypothesis) + (1 - Y) * tf.log(1 - hypothesis))
train = tf.train.GradientDescentOptimizer(learning_rate=0.1).minimize(cost)

# Accuracy computation
# True if hypothesis>0.5 else False
predicted = tf.cast(hypothesis > 0.5, dtype=tf.float32)
accuracy = tf.reduce_mean(tf.cast(tf.equal(predicted, Y), dtype=tf.float32))

# Launch graph
with tf.Session() as sess:
    # Initialize Tensorflow variables
    sess.run(tf.global_variables_initializer())
    
    for step in range(10001):
        sess.run(train, feed_dict={X: x_data, Y: y_data})
        if step % 100 == 0:
            print(step, sess.run(cost, feed_dict={X: x_data, Y: y_data}), sess.run(W))
    
    # Accuracy report
    h, c, a = sess.run([hypothesis, predicted, accuracy], feed_dict={X: x_data, Y: y_data})
    print("\nHypothesis: ", h, "\nCorrect: ", c, "\nAccuracy ", a)
In [ ]:
...

9600 0.014378563 [[ 0.63577235]
 [-0.05965849]]
9700 0.01417592 [[ 0.63577235]
 [-0.05965849]]
9800 0.013978702 [[ 0.63577235]
 [-0.05965849]]
9900 0.013786772 [[ 0.63577235]
 [-0.05965849]]
10000 0.013599854 [[ 0.63577235]
 [-0.05965849]]

Hypothesis:  [[0.01117242]
 [0.98781025]
 [0.9878056 ]
 [0.01845764]] 
Correct:  [[0.]
 [1.]
 [1.]
 [0.]] 
Accuracy  1.0
 

이번에는 더 넓게 or 더 깊게 연결 시켜볼까?

In [13]:
Image.open('wide.png')
Out[13]:
 

Wide NN for XOR

 

기존에는 입력값 w,b에 대해서 layer1에서 2개의 값을 layer2로 보냈다면, 이번에는 layer1에서 여러개의 값을 layer2로 보내보자.

In [ ]:
W1 = tf.Variable(tf.random_normal([2, 10]), name='weight1') 
b1 = tf.Variable(tf.random_normal([10]), name='bias1')
layer1 = tf.sigmoid(tf.matmul(X, W1) + b1)

W2 = tf.Variable(tf.random_normal([10, 1]), name='weight2')
b2 = tf.Variable(tf.random_normal([1]), name='bias2')
hypothesis = tf.sigmoid(tf.matmul(layer1, W2) + b2)
 

wide한 모델 구현

In [ ]:
x_data = np.array([[0, 0], [0, 1], [1, 0], [1, 1]], dtype=np.float32)
y_data = np.array([[0],    [1],    [1],    [0]], dtype=np.float32)
X = tf.placeholder(tf.float32)
Y = tf.placeholder(tf.float32)

W1 = tf.Variable(tf.random_normal([2, 10]), name='weight1')
b1 = tf.Variable(tf.random_normal([10]), name='bias1')
layer1 = tf.sigmoid(tf.matmul(X, W1) + b1)

W2 = tf.Variable(tf.random_normal([10, 1]), name='weight2')
b2 = tf.Variable(tf.random_normal([1]), name='bias2')
hypothesis = tf.sigmoid(tf.matmul(layer1, W2) + b2)

# cost/loss function
cost = -tf.reduce_mean(Y * tf.log(hypothesis) + (1 - Y) * tf.log(1 - hypothesis))
train = tf.train.GradientDescentOptimizer(learning_rate=0.1).minimize(cost)

# Accuracy computation
# True if hypothesis>0.5 else False
predicted = tf.cast(hypothesis > 0.5, dtype=tf.float32)
accuracy = tf.reduce_mean(tf.cast(tf.equal(predicted, Y), dtype=tf.float32))

# Launch graph
with tf.Session() as sess:
    # Initialize Tensorflow variables
    sess.run(tf.global_variables_initializer())
    
    for step in range(10001):
        sess.run(train, feed_dict={X: x_data, Y: y_data})
        if step % 100 == 0:
            print(step, sess.run(cost, feed_dict={X: x_data, Y: y_data}), sess.run(W))
    
    # Accuracy report
    h, c, a = sess.run([hypothesis, predicted, accuracy], feed_dict={X: x_data, Y: y_data})
    print("\nHypothesis: ", h, "\nCorrect: ", c, "\nAccuracy ", a)
In [ ]:
...

9600 0.0068609957 [[-1.2364072]
 [ 0.1998814]]
9700 0.006755276 [[-1.2364072]
 [ 0.1998814]]
9800 0.0066525377 [[-1.2364072]
 [ 0.1998814]]
9900 0.006552601 [[-1.2364072]
 [ 0.1998814]]
10000 0.0064554936 [[-1.2364072]
 [ 0.1998814]]

Hypothesis:  [[0.00535473]
 [0.99357903]
 [0.99321234]
 [0.00717448]] 
Correct:  [[0.]
 [1.]
 [1.]
 [0.]] 
Accuracy  1.0
 

Deep NN for XOR

In [ ]:
W1 = tf.Variable(tf.random_normal([2, 10]), name='weight1') 
b1 = tf.Variable(tf.random_normal([10]), name='bias1')
layer1 = tf.sigmoid(tf.matmul(X, W1) + b1)

W2 = tf.Variable(tf.random_normal([10, 10]), name='weight2')
b2 = tf.Variable(tf.random_normal([10]), name='bias2')
layer2 = tf.sigmoid(tf.matmul(layer1, W2) + b2)

W3 = tf.Variable(tf.random_normal([10, 10]), name='weight3')
b3 = tf.Variable(tf.random_normal([10]), name='bias3')
layer3 = tf.sigmoid(tf.matmul(layer2, W3) + b3)

W4 = tf.Variable(tf.random_normal([10, 1]), name='weight4')
b4 = tf.Variable(tf.random_normal([1]), name='bias4')
hypothesis = tf.sigmoid(tf.matmul(layer3, W4) + b4)
 

Deep한 모델 구현

In [ ]:
x_data = np.array([[0, 0], [0, 1], [1, 0], [1, 1]], dtype=np.float32)
y_data = np.array([[0],    [1],    [1],    [0]], dtype=np.float32)
X = tf.placeholder(tf.float32)
Y = tf.placeholder(tf.float32)

W1 = tf.Variable(tf.random_normal([2, 10]), name='weight1') 
b1 = tf.Variable(tf.random_normal([10]), name='bias1')
layer1 = tf.sigmoid(tf.matmul(X, W1) + b1)

W2 = tf.Variable(tf.random_normal([10, 10]), name='weight2')
b2 = tf.Variable(tf.random_normal([10]), name='bias2')
layer2 = tf.sigmoid(tf.matmul(layer1, W2) + b2)

W3 = tf.Variable(tf.random_normal([10, 10]), name='weight3')
b3 = tf.Variable(tf.random_normal([10]), name='bias3')
layer3 = tf.sigmoid(tf.matmul(layer2, W3) + b3)

W4 = tf.Variable(tf.random_normal([10, 1]), name='weight4')
b4 = tf.Variable(tf.random_normal([1]), name='bias4')
hypothesis = tf.sigmoid(tf.matmul(layer3, W4) + b4)

# cost/loss function
cost = -tf.reduce_mean(Y * tf.log(hypothesis) + (1 - Y) * tf.log(1 - hypothesis))
train = tf.train.GradientDescentOptimizer(learning_rate=0.1).minimize(cost)

# Accuracy computation
# True if hypothesis>0.5 else False
predicted = tf.cast(hypothesis > 0.5, dtype=tf.float32)
accuracy = tf.reduce_mean(tf.cast(tf.equal(predicted, Y), dtype=tf.float32))

# Launch graph
with tf.Session() as sess:
    # Initialize Tensorflow variables
    sess.run(tf.global_variables_initializer())
    
    for step in range(10001):
        sess.run(train, feed_dict={X: x_data, Y: y_data})
        if step % 100 == 0:
            print(step, sess.run(cost, feed_dict={X: x_data, Y: y_data}), sess.run(W))
    
    # Accuracy report
    h, c, a = sess.run([hypothesis, predicted, accuracy], feed_dict={X: x_data, Y: y_data})
    print("\nHypothesis: ", h, "\nCorrect: ", c, "\nAccuracy ", a)
In [ ]:
...

9600 0.0011915546 [[ 0.04345283]
 [-0.13949525]]
9700 0.0011749794 [[ 0.04345283]
 [-0.13949525]]
9800 0.0011588369 [[ 0.04345283]
 [-0.13949525]]
9900 0.0011431125 [[ 0.04345283]
 [-0.13949525]]
10000 0.0011277614 [[ 0.04345283]
 [-0.13949525]]

Hypothesis:  [[1.046961e-03]
 [9.986511e-01]
 [9.987583e-01]
 [8.708689e-04]] 
Correct:  [[0.]
 [1.]
 [1.]
 [0.]] 
Accuracy  1.0
 

2 layer보다 4 layer이었을때 Hypothesis가 더 빨리 0에 다가가는 것을 볼 수 있다.

 

Mnist 데이터로 실습해보기

 

출처: https://www.inflearn.com/course/%EA%B8%B0%EB%B3%B8%EC%A0%81%EC%9D%B8-%EB%A8%B8%EC%8B%A0%EB%9F%AC%EB%8B%9D-%EB%94%A5%EB%9F%AC%EB%8B%9D-%EA%B0%95%EC%A2%8C/lecture/3408

'beginner > 파이썬 딥러닝 기초' 카테고리의 다른 글

XSigmoid 보다 ReLU가 더 좋아  (0) 2019.05.08
Tensor Board로 딥네트웍 들여다보기  (0) 2019.05.07
딥 네트웍 학습 시키기  (0) 2019.05.06
XOR 문제 딥러닝으로 풀기  (0) 2019.05.06
Tensor Manipulation  (0) 2019.05.06
공지사항
최근에 올라온 글
최근에 달린 댓글
Total
Today
Yesterday
링크
TAG more
«   2024/12   »
일 월 화 수 목 금 토
1 2 3 4 5 6 7
8 9 10 11 12 13 14
15 16 17 18 19 20 21
22 23 24 25 26 27 28
29 30 31
글 보관함