[인공지능 #3 ] Cost 최소화 알고리즘 구현

 daviduino.co.kr
techtogetworld.com
 david201207.blog.me
cafe.naver.com/3dpservicedavid


인공지능 구현에 대한 글입니다.
글의 순서는 아래와 같습니다.

================================================
1. Cost 최소화 알고리즘 구현 소스코드 분석
2. Cost Minimize 코딩 작성 ( cost 를 미분해서 W변화량 적용)
3. Optimizer 적용 테스트
 - 간단한 미분은 상기과 같이 직접코딩이 가능하겠으나, 복잡해지면 통상적으로 optimizer로 코딩함
4. Optimizer 적용 과 수작업(미분계산) 수치 비교
 - 경사도 수치를 중간에 변형할수 있도록 코딩함 (gvs) ==> 거의 같음
5. Next step
  ==> multivariable linear regresstion : 하나이상의 변수적용
6. 참고자료
=================================================

[ Cost 최소화 알고리즘 구현 소스코드 분석 ]

import os

os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'

# Lab 3 Minimizing Cost

import tensorflow as tf

import matplotlib.pyplot as plt

==> matplotlib를 사용키 위해서는 별도로 설치를 해야함

==> http://matplotlib.org/users/installing.html

==> 그래프를 그려주는 기능임.

tf.set_random_seed(777)  # for reproducibility

X = [1, 2, 3]

Y = [1, 2, 3]

W = tf.placeholder(tf.float32)

# Our hypothesis for linear model X * W

hypothesis = X * W

# cost/loss function

cost = tf.reduce_mean(tf.square(hypothesis - Y))

# Launch the graph in a session.

sess = tf.Session()

# Variables for plotting cost function

W_history = []

cost_history = []

==> 그래프를 그리기 위해 배열할당

for i in range(-30, 50):

    curr_W = i * 0.1

    curr_cost = sess.run(cost, feed_dict={W: curr_W})

    W_history.append(curr_W)

    cost_history.append(curr_cost)

# Show the cost function

plt.plot(W_history, cost_history)

plt.show()

[ Cost Minimize 코딩 작성 ( cost 를 미분해서 W변화량 적용) ]

import os

os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'

# Lab 3 Minimizing Cost

import tensorflow as tf

tf.set_random_seed(777)  # for reproducibility

x_data = [1, 2, 3]

y_data = [1, 2, 3]

# Try to find values for W and b to compute y_data = W * x_data + b

# We know that W should be 1 and b should be 0

# But let's use TensorFlow to figure it out

W = tf.Variable(tf.random_normal([1]), name='weight')

X = tf.placeholder(tf.float32)

Y = tf.placeholder(tf.float32)

# Our hypothesis for linear model X * W

hypothesis = X * W

# cost/loss function

cost = tf.reduce_mean(tf.square(hypothesis - Y))

W값의  descent 간격을 구하는 과정을 미분방정식으로 직접(수작업) 코딩하는 내용입니다( 아래그림은 미분적용 과정입니다)

 

# Minimize: Gradient Descent using derivative: W -= learning_rate * derivative

learning_rate = 0.1

gradient = tf.reduce_mean((W * X - Y) * X)

descent = W - learning_rate * gradient

update = W.assign(descent)

간단한 미분은 상기과 같이 직접코딩이 가능하겠으나, 복잡해지면 통상적으로 아래와 같이 코딩을 하는것이 일반적임(텐서플로우가 자동계산을 해줌)

# Launch the graph in a session.

sess = tf.Session()

# Initializes global variables in the graph.

sess.run(tf.global_variables_initializer())

for step in range(21):

    sess.run(update, feed_dict={X: x_data, Y: y_data})

    print(step, sess.run(cost, feed_dict={X: x_data, Y: y_data}), sess.run(W))

'''

0 1.93919 [ 1.64462376]

1 0.551591 [ 1.34379935]

2 0.156897 [ 1.18335962]

3 0.0446285 [ 1.09779179]

4 0.0126943 [ 1.05215561]

5 0.00361082 [ 1.0278163]

6 0.00102708 [ 1.01483536]

7 0.000292144 [ 1.00791216]

8 8.30968e-05 [ 1.00421977]

9 2.36361e-05 [ 1.00225055]

10 6.72385e-06 [ 1.00120032]

11 1.91239e-06 [ 1.00064015]

12 5.43968e-07 [ 1.00034142]

13 1.54591e-07 [ 1.00018203]

14 4.39416e-08 [ 1.00009704]

15 1.24913e-08 [ 1.00005174]

16 3.5322e-09 [ 1.00002754]

17 9.99824e-10 [ 1.00001466]

18 2.88878e-10 [ 1.00000787]

19 8.02487e-11 [ 1.00000417]

20 2.34053e-11 [ 1.00000226]

'''

[ Optimizer 적용 테스트 ]

import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
# Lab 3 Minimizing Cost
import tensorflow as tf
tf.set_random_seed(777)  # for reproducibility

# tf Graph Input
X = [1, 2, 3]
Y = [1, 2, 3]

# Set wrong model weights
W = tf.Variable(5.0)   ==> w 값을 터무니없는 값을 주고, 적정한 값을 찾아가는지 테스트함

# Linear model
hypothesis = X * W

# cost/loss function
cost = tf.reduce_mean(tf.square(hypothesis - Y))

# Minimize: Gradient Descent Magic
optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.1)
train = optimizer.minimize(cost)

# Launch the graph in a session.
sess = tf.Session()
# Initializes global variables in the graph.
sess.run(tf.global_variables_initializer())

for step in range(100):
    print(step, sess.run(W))
    sess.run(train)

'''
0 5.0
1 1.26667
2 1.01778
3 1.00119
4 1.00008
...
96 1.0
97 1.0
98 1.0
99 1.0
'''


[ Optimizer 적용 과 수작업(미분계산) 수치 비교]

 import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'

# Lab 3 Minimizing Cost
# This is optional
import tensorflow as tf
tf.set_random_seed(777)  # for reproducibility

# tf Graph Input
X = [1, 2, 3]
Y = [1, 2, 3]

# Set wrong model weights
W = tf.Variable(5.)

# Linear model
hypothesis = X * W

# Manual gradient
gradient = tf.reduce_mean((W * X - Y) * X) * 2

# cost/loss function
cost = tf.reduce_mean(tf.square(hypothesis - Y))

# Minimize: Gradient Descent Magic
optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.01)
train = optimizer.minimize(cost)

# Get gradients
gvs = optimizer.compute_gradients(cost, [W])
# Optional: modify gradient if necessary
# gvs = [(tf.clip_by_value(grad, -1., 1.), var) for grad, var in gvs]
==> gvc 수치를 필요에 따라서 변형적용이 가능함

# Apply gradients
apply_gradients = optimizer.apply_gradients(gvs)

# Launch the graph in a session.
sess = tf.Session()
# Initializes global variables in the graph.
sess.run(tf.global_variables_initializer())

for step in range(100):
    print(step, sess.run([gradient, W, gvs]))
    sess.run(apply_gradients)
    # Same as sess.run(train)


'''
# Apply gradients
0 [37.333332, 5.0, [(37.333336, 5.0)]]
1 [33.848888, 4.6266665, [(33.848888, 4.6266665)]]
2 [30.689657, 4.2881775, [(30.689657, 4.2881775)]]
3 [27.825287, 3.9812808, [(27.825287, 3.9812808)]]
4 [25.228262, 3.703028, [(25.228264, 3.703028)]]
...
96 [0.0030694802, 1.0003289, [(0.0030694804, 1.0003289)]]
97 [0.0027837753, 1.0002983, [(0.0027837753, 1.0002983)]]
98 [0.0025234222, 1.0002704, [(0.0025234222, 1.0002704)]]
99 [0.0022875469, 1.0002451, [(0.0022875469, 1.0002451)]]
'''

[ 참고자료 ]

https://www.inflearn.com/course/기본적인-머신러닝-딥러닝-강좌/ 

http://agiantmind.tistory.com/176 

https://www.tensorflow.org/install/

https://github.com/hunkim/deeplearningzerotoall

http://www.derivative-calculator.net/

http://terms.naver.com/entry.nhn?docId=3350391&cid=58247&categoryId=58247    ==> 미분계산/공식

http://matplotlib.org/users/installing.html   ==>matplotlib 설치