[인공지능 #11 ] hello-rnn /char-seq-rnn /char-seq-softmax-only /rnn_long_char
인공지능 구현에 대한 글입니다.
글의 순서는 아래와 같습니다.
================================================
1.#lab-12-1-hello-rnn
전 단계의 출력이 다음단계의 출력에 영향을 주는 경우 적용함
- 단어, 연관검색등..
2. # lab-12-2-char-seq-rnn
-rnn 적용 ==> 정확도 높음
. 49 loss: 0.000650434 Prediction: if you want you
. y값 if you want you
3. #lab-12-3-char-seq-softmax-only
rnn 미적용 ==> 정확도 미흡함
2999 loss: 0.277323 Prediction: yf you yant you
y값 if you want you
4. # lab-12-4-rnn_long_char
error : from __future__ import print_function ==> 실행불가로 주석처리함
MultiRNNCell 로 여러단을 만들면 , 정확도가 높아짐
softmax =>reshape 수행
6. 코드탐구(추가)
==>lab-12-5-rnn_stock_prediction
lab-13-1-mnist_using_scope
lab-13-2-mnist_tensorboard
lab-13-3-mnist_save_restore
7. 참고자료
=================================================
[ #lab-12-1-hello-rnn ]
#lab-12-1-hello-rnn
"""
전 단계의 출력이 다음단계의 출력에 영향을 주는 경우 적용함
- 단어, 연관검색등..
"""
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
# Lab 12 RNN
import tensorflow as tf
import numpy as np
tf.set_random_seed(777) # reproducibility
idx2char = ['h', 'i', 'e', 'l', 'o']
# Teach hello: hihell -> ihello
x_data = [[0, 1, 0, 2, 3, 3]] # hihell
x_one_hot = [[[1, 0, 0, 0, 0], # h 0
[0, 1, 0, 0, 0], # i 1
[1, 0, 0, 0, 0], # h 0
[0, 0, 1, 0, 0], # e 2
[0, 0, 0, 1, 0], # l 3
[0, 0, 0, 1, 0]]] # l 3
y_data = [[1, 0, 2, 3, 3, 4]] # ihello
num_classes = 5
input_dim = 5 # one-hot size
hidden_size = 5 # output from the LSTM. 5 to directly predict one-hot
batch_size = 1 # one sentence
sequence_length = 6 # |ihello| == 6
learning_rate = 0.1
X = tf.placeholder(
tf.float32, [None, sequence_length, input_dim]) # X one-hot
Y = tf.placeholder(tf.int32, [None, sequence_length]) # Y label
cell = tf.contrib.rnn.BasicLSTMCell(num_units=hidden_size, state_is_tuple=True)
initial_state = cell.zero_state(batch_size, tf.float32)
outputs, _states = tf.nn.dynamic_rnn(
cell, X, initial_state=initial_state, dtype=tf.float32)
# FC layer
X_for_fc = tf.reshape(outputs, [-1, hidden_size])
# fc_w = tf.get_variable("fc_w", [hidden_size, num_classes])
# fc_b = tf.get_variable("fc_b", [num_classes])
# outputs = tf.matmul(X_for_fc, fc_w) + fc_b
outputs = tf.contrib.layers.fully_connected(
inputs=X_for_fc, num_outputs=num_classes, activation_fn=None)
# reshape out for sequence_loss
outputs = tf.reshape(outputs, [batch_size, sequence_length, num_classes])
weights = tf.ones([batch_size, sequence_length])
sequence_loss = tf.contrib.seq2seq.sequence_loss(
logits=outputs, targets=Y, weights=weights)
loss = tf.reduce_mean(sequence_loss)
train = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(loss)
prediction = tf.argmax(outputs, axis=2)
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
for i in range(50):
l, _ = sess.run([loss, train], feed_dict={X: x_one_hot, Y: y_data})
result = sess.run(prediction, feed_dict={X: x_one_hot})
print(i, "loss:", l, "prediction: ", result, "true Y: ", y_data)
# print char using dic
result_str = [idx2char[c] for c in np.squeeze(result)]
print("\tPrediction str: ", ''.join(result_str))
'''
0 loss: 1.71584 prediction: [[2 2 2 3 3 2]] true Y: [[1, 0, 2, 3, 3, 4]]
Prediction str: eeelle
1 loss: 1.56447 prediction: [[3 3 3 3 3 3]] true Y: [[1, 0, 2, 3, 3, 4]]
Prediction str: llllll
2 loss: 1.46284 prediction: [[3 3 3 3 3 3]] true Y: [[1, 0, 2, 3, 3, 4]]
Prediction str: llllll
3 loss: 1.38073 prediction: [[3 3 3 3 3 3]] true Y: [[1, 0, 2, 3, 3, 4]]
Prediction str: llllll
4 loss: 1.30603 prediction: [[3 3 3 3 3 3]] true Y: [[1, 0, 2, 3, 3, 4]]
Prediction str: llllll
5 loss: 1.21498 prediction: [[3 3 3 3 3 3]] true Y: [[1, 0, 2, 3, 3, 4]]
Prediction str: llllll
6 loss: 1.1029 prediction: [[3 0 3 3 3 4]] true Y: [[1, 0, 2, 3, 3, 4]]
Prediction str: lhlllo
7 loss: 0.982386 prediction: [[1 0 3 3 3 4]] true Y: [[1, 0, 2, 3, 3, 4]]
Prediction str: ihlllo
8 loss: 0.871259 prediction: [[1 0 3 3 3 4]] true Y: [[1, 0, 2, 3, 3, 4]]
Prediction str: ihlllo
9 loss: 0.774338 prediction: [[1 0 2 3 3 4]] true Y: [[1, 0, 2, 3, 3, 4]]
Prediction str: ihello
10 loss: 0.676005 prediction: [[1 0 2 3 3 4]] true Y: [[1, 0, 2, 3, 3, 4]]
Prediction str: ihello
...
'''
[# lab-12-2-char-seq-rnn ]
# lab-12-2-char-seq-rnn
"""
rnn 적용 ==> 정확도 높음
- 49 loss: 0.000650434 Prediction: if you want you
- y값 if you want you
"""
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
# Lab 12 Character Sequence RNN
import tensorflow as tf
import numpy as np
tf.set_random_seed(777) # reproducibility
sample = " if you want you"
idx2char = list(set(sample)) # index -> char
char2idx = {c: i for i, c in enumerate(idx2char)} # char -> idex
# hyper parameters
dic_size = len(char2idx) # RNN input size (one hot size)
hidden_size = len(char2idx) # RNN output size
num_classes = len(char2idx) # final output size (RNN or softmax, etc.)
batch_size = 1 # one sample data, one batch
sequence_length = len(sample) - 1 # number of lstm rollings (unit #)
learning_rate = 0.1
sample_idx = [char2idx[c] for c in sample] # char to index
x_data = [sample_idx[:-1]] # X data sample (0 ~ n-1) hello: hell
y_data = [sample_idx[1:]] # Y label sample (1 ~ n) hello: ello
X = tf.placeholder(tf.int32, [None, sequence_length]) # X data
Y = tf.placeholder(tf.int32, [None, sequence_length]) # Y label
x_one_hot = tf.one_hot(X, num_classes) # one hot: 1 -> 0 1 0 0 0 0 0 0 0 0
cell = tf.contrib.rnn.BasicLSTMCell(
num_units=hidden_size, state_is_tuple=True)
initial_state = cell.zero_state(batch_size, tf.float32)
outputs, _states = tf.nn.dynamic_rnn(
cell, x_one_hot, initial_state=initial_state, dtype=tf.float32)
# FC layer
X_for_fc = tf.reshape(outputs, [-1, hidden_size])
outputs = tf.contrib.layers.fully_connected(X_for_fc, num_classes, activation_fn=None)
# reshape out for sequence_loss
outputs = tf.reshape(outputs, [batch_size, sequence_length, num_classes])
weights = tf.ones([batch_size, sequence_length])
sequence_loss = tf.contrib.seq2seq.sequence_loss(
logits=outputs, targets=Y, weights=weights)
loss = tf.reduce_mean(sequence_loss)
train = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(loss)
prediction = tf.argmax(outputs, axis=2)
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
for i in range(50):
l, _ = sess.run([loss, train], feed_dict={X: x_data, Y: y_data})
result = sess.run(prediction, feed_dict={X: x_data})
# print char using dic
result_str = [idx2char[c] for c in np.squeeze(result)]
print(i, "loss:", l, "Prediction:", ''.join(result_str))
'''
0 loss: 2.35377 Prediction: uuuuuuuuuuuuuuu
1 loss: 2.21383 Prediction: yy you y you
2 loss: 2.04317 Prediction: yy yoo ou
3 loss: 1.85869 Prediction: yy ou uou
4 loss: 1.65096 Prediction: yy you a you
5 loss: 1.40243 Prediction: yy you yan you
6 loss: 1.12986 Prediction: yy you wann you
7 loss: 0.907699 Prediction: yy you want you
8 loss: 0.687401 Prediction: yf you want you
9 loss: 0.508868 Prediction: yf you want you
10 loss: 0.379423 Prediction: yf you want you
11 loss: 0.282956 Prediction: if you want you
12 loss: 0.208561 Prediction: if you want you
...
'''
#lab-12-3-char-seq-softmax-only
"""
rnn 미적용 ==> 정확도 미흡함
- 2999 loss: 0.277323 Prediction: yf you yant you
- y값 if you want you
"""
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
# Lab 12 Character Sequence Softmax only
import tensorflow as tf
import numpy as np
tf.set_random_seed(777) # reproducibility
sample = " if you want you"
idx2char = list(set(sample)) # index -> char
char2idx = {c: i for i, c in enumerate(idx2char)} # char -> idex
# hyper parameters
dic_size = len(char2idx) # RNN input size (one hot size)
rnn_hidden_size = len(char2idx) # RNN output size
num_classes = len(char2idx) # final output size (RNN or softmax, etc.)
batch_size = 1 # one sample data, one batch
sequence_length = len(sample) - 1 # number of lstm rollings (unit #)
learning_rate = 0.1
sample_idx = [char2idx[c] for c in sample] # char to index
x_data = [sample_idx[:-1]] # X data sample (0 ~ n-1) hello: hell
y_data = [sample_idx[1:]] # Y label sample (1 ~ n) hello: ello
X = tf.placeholder(tf.int32, [None, sequence_length]) # X data
Y = tf.placeholder(tf.int32, [None, sequence_length]) # Y label
# flatten the data (ignore batches for now). No effect if the batch size is 1
X_one_hot = tf.one_hot(X, num_classes) # one hot: 1 -> 0 1 0 0 0 0 0 0 0 0
X_for_softmax = tf.reshape(X_one_hot, [-1, rnn_hidden_size])
# softmax layer (rnn_hidden_size -> num_classes)
softmax_w = tf.get_variable("softmax_w", [rnn_hidden_size, num_classes])
softmax_b = tf.get_variable("softmax_b", [num_classes])
outputs = tf.matmul(X_for_softmax, softmax_w) + softmax_b
# expend the data (revive the batches)
outputs = tf.reshape(outputs, [batch_size, sequence_length, num_classes])
weights = tf.ones([batch_size, sequence_length])
# Compute sequence cost/loss
sequence_loss = tf.contrib.seq2seq.sequence_loss(
logits=outputs, targets=Y, weights=weights)
loss = tf.reduce_mean(sequence_loss) # mean all sequence loss
train = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(loss)
prediction = tf.argmax(outputs, axis=2)
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
for i in range(3000):
l, _ = sess.run([loss, train], feed_dict={X: x_data, Y: y_data})
result = sess.run(prediction, feed_dict={X: x_data})
# print char using dic
result_str = [idx2char[c] for c in np.squeeze(result)]
print(i, "loss:", l, "Prediction:", ''.join(result_str))
'''
0 loss: 2.29513 Prediction: yu yny y y oyny
1 loss: 2.10156 Prediction: yu ynu y y oynu
2 loss: 1.92344 Prediction: yu you y u you
..
2997 loss: 0.277323 Prediction: yf you yant you
2998 loss: 0.277323 Prediction: yf you yant you
2999 loss: 0.277323 Prediction: yf you yant you
'''
[# lab-12-4-rnn_long_char]
# lab-12-4-rnn_long_char
"""
error : from __future__ import print_function ==> 실행불가로 주석처리함
# MultiRNNCell 로 여러단을 만들면 , 정확도가 높아짐
# softmax =>reshape 수행
"""
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
# from __future__ import print_function
import tensorflow as tf
import numpy as np
from tensorflow.contrib import rnn
tf.set_random_seed(777) # reproducibility
sentence = ("if you want to build a ship, don't drum up people together to "
"collect wood and don't assign them tasks and work, but rather "
"teach them to long for the endless immensity of the sea.")
char_set = list(set(sentence))
char_dic = {w: i for i, w in enumerate(char_set)}
data_dim = len(char_set)
hidden_size = len(char_set)
num_classes = len(char_set)
sequence_length = 10 # Any arbitrary number
learning_rate = 0.1
dataX = []
dataY = []
for i in range(0, len(sentence) - sequence_length):
x_str = sentence[i:i + sequence_length]
y_str = sentence[i + 1: i + sequence_length + 1]
print(i, x_str, '->', y_str)
x = [char_dic[c] for c in x_str] # x str to index
y = [char_dic[c] for c in y_str] # y str to index
dataX.append(x)
dataY.append(y)
batch_size = len(dataX)
X = tf.placeholder(tf.int32, [None, sequence_length])
Y = tf.placeholder(tf.int32, [None, sequence_length])
# One-hot encoding
X_one_hot = tf.one_hot(X, num_classes)
print(X_one_hot) # check out the shape
# Make a lstm cell with hidden_size (each unit output vector size)
def lstm_cell():
cell = rnn.BasicLSTMCell(hidden_size, state_is_tuple=True)
return cell
multi_cells = rnn.MultiRNNCell([lstm_cell() for _ in range(2)], state_is_tuple=True)
# 위와 같이.MultiRNNCell 로 여러단을 만들면 , 정확도가 높아짐
# outputs: unfolding size x hidden size, state = hidden size
outputs, _states = tf.nn.dynamic_rnn(multi_cells, X_one_hot, dtype=tf.float32)
# softmax =>reshape 수행
# FC layer
X_for_fc = tf.reshape(outputs, [-1, hidden_size])
outputs = tf.contrib.layers.fully_connected(X_for_fc, num_classes, activation_fn=None)
# reshape out for sequence_loss
outputs = tf.reshape(outputs, [batch_size, sequence_length, num_classes])
# All weights are 1 (equal weights)
weights = tf.ones([batch_size, sequence_length])
sequence_loss = tf.contrib.seq2seq.sequence_loss(
logits=outputs, targets=Y, weights=weights)
mean_loss = tf.reduce_mean(sequence_loss)
train_op = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(mean_loss)
sess = tf.Session()
sess.run(tf.global_variables_initializer())
for i in range(500):
_, l, results = sess.run(
[train_op, mean_loss, outputs], feed_dict={X: dataX, Y: dataY})
for j, result in enumerate(results):
index = np.argmax(result, axis=1)
print(i, j, ''.join([char_set[t] for t in index]), l)
# Let's print the last char of each result to check it works
results = sess.run(outputs, feed_dict={X: dataX})
for j, result in enumerate(results):
index = np.argmax(result, axis=1)
if j is 0: # print all for the first result to make a sentence
print(''.join([char_set[t] for t in index]), end='')
else:
print(char_set[index[-1]], end='')
'''
0 167 tttttttttt 3.23111
0 168 tttttttttt 3.23111
0 169 tttttttttt 3.23111
…
499 167 of the se 0.229616
499 168 tf the sea 0.229616
499 169 the sea. 0.229616
g you want to build a ship, don't drum up people together to collect wood and don't assign them tasks and work, but rather teach them to long for the endless immensity of the sea.
'''
[# lab-12-5-rnn_stock_prediction]
[참고자료]
https://www.inflearn.com/course/기본적인-머신러닝-딥러닝-강좌
https://github.com/hunkim/deeplearningzerotoall
https://www.tensorflow.org/api_docs/python/tf/layers
'프로젝트 > 인공지능' 카테고리의 다른 글
| [인공지능 #13 ] q_net_frozenlake / cartpole (0) | 2017.08.07 |
|---|---|
| [인공지능 #12 ] Q-Learning / OpenAI gym / frozenlake (0) | 2017.08.06 |
| [인공지능 #10 ]mnist_cnn/mnist_deep_cnn/mnist_cnn_class/mnist_cnn_layers/mnist_cnn_ensemble_layers (0) | 2017.08.06 |
| [인공지능 #9]mnist_softmax /mnist_nn/mnist_nn_xavier/mnist_nn_deep / mnist_nn_dropout (0) | 2017.08.06 |
| [인공지능 #8]xor / xor-nn / xor-nn-wide-deep (0) | 2017.08.06 |
[인공지능 #10 ]mnist_cnn/mnist_deep_cnn/mnist_cnn_class/mnist_cnn_layers/mnist_cnn_ensemble_layers
인공지능 구현에 대한 글입니다.
글의 순서는 아래와 같습니다.
================================================
1.# lab-11-1-mnist_cnn
==> http://cs.stanford.edu/people/karpathy/convnetjs/demo/cifar10.html : cnn 시물레이션
==> 2단 CNN => 정확도 98.83%
==> CNN은 수행시간이 많이 소요됨, 코드 테스트시에는 TRAINING EPOCHS를 줄여 실행할 필요가 있음.
물론 구글, 아마존 크라우드 서비스 활용도 가능함( 경우에 따라 비용발생)
그래픽 카드 (GPU NVIDIA )를 적용하는 방법도 있음.
2. # lab-11-2-mnist_deep_cnn
==> 3단 CNN : 정확도 99.38 %
3. # lab-11-3-mnist_cnn_class
==> python의 clss를 이용해서 코드의 반복을 줄이고, 코드를 단순화함
4. #lab-11-4-mnist_cnn_layers
==> tf.layers 패키지 사용하면 코드가 더욱 단순화 될수 있음
==> https://www.tensorflow.org/api_docs/python/tf/layers
5. # lab-11-5-mnist_cnn_ensemble_layers
==> 각각 예측시키고, 후에 병합을 해서 예측치를 출력시킴
==> 정확도 : 99.52%
6. 코드탐구(추가)
==>lab-11-X-mnist_cnn_low_memory
7. 참고자료
=================================================
[ lab-11-1-mnist_cnn ]
# lab-11-1-mnist_cnn
"""http://cs.stanford.edu/people/karpathy/convnetjs/demo/cifar10.html : cnn 시물레이션
2단 CNN => 정확도 98.83%
"""
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
# Lab 11 MNIST and Convolutional Neural Network
import tensorflow as tf
import random
# import matplotlib.pyplot as plt
from tensorflow.examples.tutorials.mnist import input_data
tf.set_random_seed(777) # reproducibility
mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)
# Check out https://www.tensorflow.org/get_started/mnist/beginners for
# more information about the mnist dataset
# hyper parameters
learning_rate = 0.001
training_epochs = 15
batch_size = 100
# input place holders
X = tf.placeholder(tf.float32, [None, 784])
X_img = tf.reshape(X, [-1, 28, 28, 1]) # img 28x28x1 (black/white)
Y = tf.placeholder(tf.float32, [None, 10])
# L1 ImgIn shape=(?, 28, 28, 1)
W1 = tf.Variable(tf.random_normal([3, 3, 1, 32], stddev=0.01))
# Conv -> (?, 28, 28, 32)
# Pool -> (?, 14, 14, 32)
L1 = tf.nn.conv2d(X_img, W1, strides=[1, 1, 1, 1], padding='SAME')
L1 = tf.nn.relu(L1)
L1 = tf.nn.max_pool(L1, ksize=[1, 2, 2, 1],
strides=[1, 2, 2, 1], padding='SAME')
'''
Tensor("Conv2D:0", shape=(?, 28, 28, 32), dtype=float32)
Tensor("Relu:0", shape=(?, 28, 28, 32), dtype=float32)
Tensor("MaxPool:0", shape=(?, 14, 14, 32), dtype=float32)
'''
# L2 ImgIn shape=(?, 14, 14, 32)
W2 = tf.Variable(tf.random_normal([3, 3, 32, 64], stddev=0.01))
# Conv ->(?, 14, 14, 64)
# Pool ->(?, 7, 7, 64)
L2 = tf.nn.conv2d(L1, W2, strides=[1, 1, 1, 1], padding='SAME')
L2 = tf.nn.relu(L2)
L2 = tf.nn.max_pool(L2, ksize=[1, 2, 2, 1],
strides=[1, 2, 2, 1], padding='SAME')
L2_flat = tf.reshape(L2, [-1, 7 * 7 * 64])
'''
Tensor("Conv2D_1:0", shape=(?, 14, 14, 64), dtype=float32)
Tensor("Relu_1:0", shape=(?, 14, 14, 64), dtype=float32)
Tensor("MaxPool_1:0", shape=(?, 7, 7, 64), dtype=float32)
Tensor("Reshape_1:0", shape=(?, 3136), dtype=float32)
'''
# Final FC 7x7x64 inputs -> 10 outputs
W3 = tf.get_variable("W3", shape=[7 * 7 * 64, 10],
initializer=tf.contrib.layers.xavier_initializer())
b = tf.Variable(tf.random_normal([10]))
logits = tf.matmul(L2_flat, W3) + b
# define cost/loss & optimizer
cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(
logits=logits, labels=Y))
optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(cost)
# initialize
sess = tf.Session()
sess.run(tf.global_variables_initializer())
# train my model
print('Learning started. It takes sometime.')
for epoch in range(training_epochs):
avg_cost = 0
total_batch = int(mnist.train.num_examples / batch_size)
for i in range(total_batch):
batch_xs, batch_ys = mnist.train.next_batch(batch_size)
feed_dict = {X: batch_xs, Y: batch_ys}
c, _ = sess.run([cost, optimizer], feed_dict=feed_dict)
avg_cost += c / total_batch
print('Epoch:', '%04d' % (epoch + 1), 'cost =', '{:.9f}'.format(avg_cost))
print('Learning Finished!')
# Test model and check accuracy
correct_prediction = tf.equal(tf.argmax(logits, 1), tf.argmax(Y, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
print('Accuracy:', sess.run(accuracy, feed_dict={
X: mnist.test.images, Y: mnist.test.labels}))
# Get one and predict
r = random.randint(0, mnist.test.num_examples - 1)
print("Label: ", sess.run(tf.argmax(mnist.test.labels[r:r + 1], 1)))
print("Prediction: ", sess.run(
tf.argmax(logits, 1), feed_dict={X: mnist.test.images[r:r + 1]}))
# plt.imshow(mnist.test.images[r:r + 1].
# reshape(28, 28), cmap='Greys', interpolation='nearest')
# plt.show()
'''
Epoch: 0001 cost = 0.340291267
Epoch: 0002 cost = 0.090731326
Epoch: 0003 cost = 0.064477619
Epoch: 0004 cost = 0.050683064
Epoch: 0005 cost = 0.041864835
Epoch: 0006 cost = 0.035760704
Epoch: 0007 cost = 0.030572132
Epoch: 0008 cost = 0.026207981
Epoch: 0009 cost = 0.022622454
Epoch: 0010 cost = 0.019055919
Epoch: 0011 cost = 0.017758641
Epoch: 0012 cost = 0.014156652
Epoch: 0013 cost = 0.012397016
Epoch: 0014 cost = 0.010693789
Epoch: 0015 cost = 0.009469977
Learning Finished!
Accuracy: 0.9885
'''
[ # lab-11-2-mnist_deep_cnn ]
# lab-11-2-mnist_deep_cnn
"""
3단 CNN : 정확도 99.38 %
"""
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
# Lab 11 MNIST and Deep learning CNN
import tensorflow as tf
import random
# import matplotlib.pyplot as plt
from tensorflow.examples.tutorials.mnist import input_data
tf.set_random_seed(777) # reproducibility
mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)
# Check out https://www.tensorflow.org/get_started/mnist/beginners for
# more information about the mnist dataset
# hyper parameters
learning_rate = 0.001
training_epochs = 15
batch_size = 100
# dropout (keep_prob) rate 0.7~0.5 on training, but should be 1 for testing
keep_prob = tf.placeholder(tf.float32)
# input place holders
X = tf.placeholder(tf.float32, [None, 784])
X_img = tf.reshape(X, [-1, 28, 28, 1]) # img 28x28x1 (black/white)
Y = tf.placeholder(tf.float32, [None, 10])
# L1 ImgIn shape=(?, 28, 28, 1)
W1 = tf.Variable(tf.random_normal([3, 3, 1, 32], stddev=0.01))
# Conv -> (?, 28, 28, 32)
# Pool -> (?, 14, 14, 32)
L1 = tf.nn.conv2d(X_img, W1, strides=[1, 1, 1, 1], padding='SAME')
L1 = tf.nn.relu(L1)
L1 = tf.nn.max_pool(L1, ksize=[1, 2, 2, 1],
strides=[1, 2, 2, 1], padding='SAME')
L1 = tf.nn.dropout(L1, keep_prob=keep_prob)
'''
Tensor("Conv2D:0", shape=(?, 28, 28, 32), dtype=float32)
Tensor("Relu:0", shape=(?, 28, 28, 32), dtype=float32)
Tensor("MaxPool:0", shape=(?, 14, 14, 32), dtype=float32)
Tensor("dropout/mul:0", shape=(?, 14, 14, 32), dtype=float32)
'''
# L2 ImgIn shape=(?, 14, 14, 32)
W2 = tf.Variable(tf.random_normal([3, 3, 32, 64], stddev=0.01))
# Conv ->(?, 14, 14, 64)
# Pool ->(?, 7, 7, 64)
L2 = tf.nn.conv2d(L1, W2, strides=[1, 1, 1, 1], padding='SAME')
L2 = tf.nn.relu(L2)
L2 = tf.nn.max_pool(L2, ksize=[1, 2, 2, 1],
strides=[1, 2, 2, 1], padding='SAME')
L2 = tf.nn.dropout(L2, keep_prob=keep_prob)
'''
Tensor("Conv2D_1:0", shape=(?, 14, 14, 64), dtype=float32)
Tensor("Relu_1:0", shape=(?, 14, 14, 64), dtype=float32)
Tensor("MaxPool_1:0", shape=(?, 7, 7, 64), dtype=float32)
Tensor("dropout_1/mul:0", shape=(?, 7, 7, 64), dtype=float32)
'''
# L3 ImgIn shape=(?, 7, 7, 64)
W3 = tf.Variable(tf.random_normal([3, 3, 64, 128], stddev=0.01))
# Conv ->(?, 7, 7, 128)
# Pool ->(?, 4, 4, 128)
# Reshape ->(?, 4 * 4 * 128) # Flatten them for FC
L3 = tf.nn.conv2d(L2, W3, strides=[1, 1, 1, 1], padding='SAME')
L3 = tf.nn.relu(L3)
L3 = tf.nn.max_pool(L3, ksize=[1, 2, 2, 1], strides=[
1, 2, 2, 1], padding='SAME')
L3 = tf.nn.dropout(L3, keep_prob=keep_prob)
L3_flat = tf.reshape(L3, [-1, 128 * 4 * 4])
'''
Tensor("Conv2D_2:0", shape=(?, 7, 7, 128), dtype=float32)
Tensor("Relu_2:0", shape=(?, 7, 7, 128), dtype=float32)
Tensor("MaxPool_2:0", shape=(?, 4, 4, 128), dtype=float32)
Tensor("dropout_2/mul:0", shape=(?, 4, 4, 128), dtype=float32)
Tensor("Reshape_1:0", shape=(?, 2048), dtype=float32)
'''
# L4 FC 4x4x128 inputs -> 625 outputs
W4 = tf.get_variable("W4", shape=[128 * 4 * 4, 625],
initializer=tf.contrib.layers.xavier_initializer())
b4 = tf.Variable(tf.random_normal([625]))
L4 = tf.nn.relu(tf.matmul(L3_flat, W4) + b4)
L4 = tf.nn.dropout(L4, keep_prob=keep_prob)
'''
Tensor("Relu_3:0", shape=(?, 625), dtype=float32)
Tensor("dropout_3/mul:0", shape=(?, 625), dtype=float32)
'''
# L5 Final FC 625 inputs -> 10 outputs
W5 = tf.get_variable("W5", shape=[625, 10],
initializer=tf.contrib.layers.xavier_initializer())
b5 = tf.Variable(tf.random_normal([10]))
logits = tf.matmul(L4, W5) + b5
'''
Tensor("add_1:0", shape=(?, 10), dtype=float32)
'''
# define cost/loss & optimizer
cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(
logits=logits, labels=Y))
optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(cost)
# initialize
sess = tf.Session()
sess.run(tf.global_variables_initializer())
# train my model
print('Learning started. It takes sometime.')
for epoch in range(training_epochs):
avg_cost = 0
total_batch = int(mnist.train.num_examples / batch_size)
for i in range(total_batch):
batch_xs, batch_ys = mnist.train.next_batch(batch_size)
feed_dict = {X: batch_xs, Y: batch_ys, keep_prob: 0.7}
c, _ = sess.run([cost, optimizer], feed_dict=feed_dict)
avg_cost += c / total_batch
print('Epoch:', '%04d' % (epoch + 1), 'cost =', '{:.9f}'.format(avg_cost))
print('Learning Finished!')
# Test model and check accuracy
# if you have a OOM error, please refer to lab-11-X-mnist_deep_cnn_low_memory.py
correct_prediction = tf.equal(tf.argmax(logits, 1), tf.argmax(Y, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
print('Accuracy:', sess.run(accuracy, feed_dict={
X: mnist.test.images, Y: mnist.test.labels, keep_prob: 1}))
# Get one and predict
r = random.randint(0, mnist.test.num_examples - 1)
print("Label: ", sess.run(tf.argmax(mnist.test.labels[r:r + 1], 1)))
print("Prediction: ", sess.run(
tf.argmax(logits, 1), feed_dict={X: mnist.test.images[r:r + 1], keep_prob: 1}))
# plt.imshow(mnist.test.images[r:r + 1].
# reshape(28, 28), cmap='Greys', interpolation='nearest')
# plt.show()
'''
Learning stared. It takes sometime.
Epoch: 0001 cost = 0.385748474
Epoch: 0002 cost = 0.092017397
Epoch: 0003 cost = 0.065854684
Epoch: 0004 cost = 0.055604566
Epoch: 0005 cost = 0.045996377
Epoch: 0006 cost = 0.040913645
Epoch: 0007 cost = 0.036924479
Epoch: 0008 cost = 0.032808939
Epoch: 0009 cost = 0.031791007
Epoch: 0010 cost = 0.030224456
Epoch: 0011 cost = 0.026849916
Epoch: 0012 cost = 0.026826763
Epoch: 0013 cost = 0.027188021
Epoch: 0014 cost = 0.023604777
Epoch: 0015 cost = 0.024607201
Learning Finished!
Accuracy: 0.9938
[ # lab-11-3-mnist_cnn_class ]
# lab-11-3-mnist_cnn_class
""" python의 clss를 이용해서 코드의 반복을 줄이고, 코드를 단순화함"""
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
# Lab 11 MNIST and Deep learning CNN
import tensorflow as tf
# import matplotlib.pyplot as plt
from tensorflow.examples.tutorials.mnist import input_data
tf.set_random_seed(777) # reproducibility
mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)
# Check out https://www.tensorflow.org/get_started/mnist/beginners for
# more information about the mnist dataset
# hyper parameters
learning_rate = 0.001
training_epochs = 15
batch_size = 100
class Model:
def __init__(self, sess, name):
self.sess = sess
self.name = name
self._build_net()
def _build_net(self):
with tf.variable_scope(self.name):
# dropout (keep_prob) rate 0.7~0.5 on training, but should be 1
# for testing
self.keep_prob = tf.placeholder(tf.float32)
# input place holders
self.X = tf.placeholder(tf.float32, [None, 784])
# img 28x28x1 (black/white)
X_img = tf.reshape(self.X, [-1, 28, 28, 1])
self.Y = tf.placeholder(tf.float32, [None, 10])
# L1 ImgIn shape=(?, 28, 28, 1)
W1 = tf.Variable(tf.random_normal([3, 3, 1, 32], stddev=0.01))
# Conv -> (?, 28, 28, 32)
# Pool -> (?, 14, 14, 32)
L1 = tf.nn.conv2d(X_img, W1, strides=[1, 1, 1, 1], padding='SAME')
L1 = tf.nn.relu(L1)
L1 = tf.nn.max_pool(L1, ksize=[1, 2, 2, 1],
strides=[1, 2, 2, 1], padding='SAME')
L1 = tf.nn.dropout(L1, keep_prob=self.keep_prob)
'''
Tensor("Conv2D:0", shape=(?, 28, 28, 32), dtype=float32)
Tensor("Relu:0", shape=(?, 28, 28, 32), dtype=float32)
Tensor("MaxPool:0", shape=(?, 14, 14, 32), dtype=float32)
Tensor("dropout/mul:0", shape=(?, 14, 14, 32), dtype=float32)
'''
# L2 ImgIn shape=(?, 14, 14, 32)
W2 = tf.Variable(tf.random_normal([3, 3, 32, 64], stddev=0.01))
# Conv ->(?, 14, 14, 64)
# Pool ->(?, 7, 7, 64)
L2 = tf.nn.conv2d(L1, W2, strides=[1, 1, 1, 1], padding='SAME')
L2 = tf.nn.relu(L2)
L2 = tf.nn.max_pool(L2, ksize=[1, 2, 2, 1],
strides=[1, 2, 2, 1], padding='SAME')
L2 = tf.nn.dropout(L2, keep_prob=self.keep_prob)
'''
Tensor("Conv2D_1:0", shape=(?, 14, 14, 64), dtype=float32)
Tensor("Relu_1:0", shape=(?, 14, 14, 64), dtype=float32)
Tensor("MaxPool_1:0", shape=(?, 7, 7, 64), dtype=float32)
Tensor("dropout_1/mul:0", shape=(?, 7, 7, 64), dtype=float32)
'''
# L3 ImgIn shape=(?, 7, 7, 64)
W3 = tf.Variable(tf.random_normal([3, 3, 64, 128], stddev=0.01))
# Conv ->(?, 7, 7, 128)
# Pool ->(?, 4, 4, 128)
# Reshape ->(?, 4 * 4 * 128) # Flatten them for FC
L3 = tf.nn.conv2d(L2, W3, strides=[1, 1, 1, 1], padding='SAME')
L3 = tf.nn.relu(L3)
L3 = tf.nn.max_pool(L3, ksize=[1, 2, 2, 1], strides=[
1, 2, 2, 1], padding='SAME')
L3 = tf.nn.dropout(L3, keep_prob=self.keep_prob)
L3_flat = tf.reshape(L3, [-1, 128 * 4 * 4])
'''
Tensor("Conv2D_2:0", shape=(?, 7, 7, 128), dtype=float32)
Tensor("Relu_2:0", shape=(?, 7, 7, 128), dtype=float32)
Tensor("MaxPool_2:0", shape=(?, 4, 4, 128), dtype=float32)
Tensor("dropout_2/mul:0", shape=(?, 4, 4, 128), dtype=float32)
Tensor("Reshape_1:0", shape=(?, 2048), dtype=float32)
'''
# L4 FC 4x4x128 inputs -> 625 outputs
W4 = tf.get_variable("W4", shape=[128 * 4 * 4, 625],
initializer=tf.contrib.layers.xavier_initializer())
b4 = tf.Variable(tf.random_normal([625]))
L4 = tf.nn.relu(tf.matmul(L3_flat, W4) + b4)
L4 = tf.nn.dropout(L4, keep_prob=self.keep_prob)
'''
Tensor("Relu_3:0", shape=(?, 625), dtype=float32)
Tensor("dropout_3/mul:0", shape=(?, 625), dtype=float32)
'''
# L5 Final FC 625 inputs -> 10 outputs
W5 = tf.get_variable("W5", shape=[625, 10],
initializer=tf.contrib.layers.xavier_initializer())
b5 = tf.Variable(tf.random_normal([10]))
self.logits = tf.matmul(L4, W5) + b5
'''
Tensor("add_1:0", shape=(?, 10), dtype=float32)
'''
# define cost/loss & optimizer
self.cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(
logits=self.logits, labels=self.Y))
self.optimizer = tf.train.AdamOptimizer(
learning_rate=learning_rate).minimize(self.cost)
correct_prediction = tf.equal(
tf.argmax(self.logits, 1), tf.argmax(self.Y, 1))
self.accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
def predict(self, x_test, keep_prop=1.0):
return self.sess.run(self.logits, feed_dict={self.X: x_test, self.keep_prob: keep_prop})
def get_accuracy(self, x_test, y_test, keep_prop=1.0):
return self.sess.run(self.accuracy, feed_dict={self.X: x_test, self.Y: y_test, self.keep_prob: keep_prop})
def train(self, x_data, y_data, keep_prop=0.7):
return self.sess.run([self.cost, self.optimizer], feed_dict={
self.X: x_data, self.Y: y_data, self.keep_prob: keep_prop})
# initialize
sess = tf.Session()
m1 = Model(sess, "m1")
sess.run(tf.global_variables_initializer())
print('Learning Started!')
# train my model
for epoch in range(training_epochs):
avg_cost = 0
total_batch = int(mnist.train.num_examples / batch_size)
for i in range(total_batch):
batch_xs, batch_ys = mnist.train.next_batch(batch_size)
c, _ = m1.train(batch_xs, batch_ys)
avg_cost += c / total_batch
print('Epoch:', '%04d' % (epoch + 1), 'cost =', '{:.9f}'.format(avg_cost))
print('Learning Finished!')
# Test model and check accuracy
print('Accuracy:', m1.get_accuracy(mnist.test.images, mnist.test.labels))
[#lab-11-4-mnist_cnn_layers]
#lab-11-4-mnist_cnn_layers
"""
tf.layers 패키지 사용하면 코드가 더욱 단순화 될수 있음
https://www.tensorflow.org/api_docs/python/tf/layers
CNN은 수행시간이 많이 소요됨, 코드 테스트시에는 TRAINING EPOCHS를 줄여 실행할 필요가 있음.
물론 구글, 아마존 크라우드 서비스 활용도 가능함( 경우에 따라 비용발생)
그래픽 카드 (GPU NVIDIA )를 적용하는 방법도 있음.
"""
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
# Lab 11 MNIST and Deep learning CNN
import tensorflow as tf
# import matplotlib.pyplot as plt
from tensorflow.examples.tutorials.mnist import input_data
tf.set_random_seed(777) # reproducibility
mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)
# Check out https://www.tensorflow.org/get_started/mnist/beginners for
# more information about the mnist dataset
# hyper parameters
learning_rate = 0.001
training_epochs = 15
batch_size = 100
class Model:
def __init__(self, sess, name):
self.sess = sess
self.name = name
self._build_net()
def _build_net(self):
with tf.variable_scope(self.name):
# dropout (keep_prob) rate 0.7~0.5 on training, but should be 1
# for testing
self.training = tf.placeholder(tf.bool)
# input place holders
self.X = tf.placeholder(tf.float32, [None, 784])
# img 28x28x1 (black/white), Input Layer
X_img = tf.reshape(self.X, [-1, 28, 28, 1])
self.Y = tf.placeholder(tf.float32, [None, 10])
# Convolutional Layer #1
conv1 = tf.layers.conv2d(inputs=X_img, filters=32, kernel_size=[3, 3],
padding="SAME", activation=tf.nn.relu)
# Pooling Layer #1
pool1 = tf.layers.max_pooling2d(inputs=conv1, pool_size=[2, 2],
padding="SAME", strides=2)
dropout1 = tf.layers.dropout(inputs=pool1,
rate=0.7, training=self.training)
# Convolutional Layer #2 and Pooling Layer #2
conv2 = tf.layers.conv2d(inputs=dropout1, filters=64, kernel_size=[3, 3],
padding="SAME", activation=tf.nn.relu)
pool2 = tf.layers.max_pooling2d(inputs=conv2, pool_size=[2, 2],
padding="SAME", strides=2)
dropout2 = tf.layers.dropout(inputs=pool2,
rate=0.7, training=self.training)
# Convolutional Layer #2 and Pooling Layer #2
conv3 = tf.layers.conv2d(inputs=dropout2, filters=128, kernel_size=[3, 3],
padding="same", activation=tf.nn.relu)
pool3 = tf.layers.max_pooling2d(inputs=conv3, pool_size=[2, 2],
padding="same", strides=2)
dropout3 = tf.layers.dropout(inputs=pool3,
rate=0.7, training=self.training)
# Dense Layer with Relu
flat = tf.reshape(dropout3, [-1, 128 * 4 * 4])
dense4 = tf.layers.dense(inputs=flat,
units=625, activation=tf.nn.relu)
dropout4 = tf.layers.dropout(inputs=dense4,
rate=0.5, training=self.training)
# Logits (no activation) Layer: L5 Final FC 625 inputs -> 10 outputs
self.logits = tf.layers.dense(inputs=dropout4, units=10)
# define cost/loss & optimizer
self.cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(
logits=self.logits, labels=self.Y))
self.optimizer = tf.train.AdamOptimizer(
learning_rate=learning_rate).minimize(self.cost)
correct_prediction = tf.equal(
tf.argmax(self.logits, 1), tf.argmax(self.Y, 1))
self.accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
def predict(self, x_test, training=False):
return self.sess.run(self.logits,
feed_dict={self.X: x_test, self.training: training})
def get_accuracy(self, x_test, y_test, training=False):
return self.sess.run(self.accuracy,
feed_dict={self.X: x_test,
self.Y: y_test, self.training: training})
def train(self, x_data, y_data, training=True):
return self.sess.run([self.cost, self.optimizer], feed_dict={
self.X: x_data, self.Y: y_data, self.training: training})
# initialize
sess = tf.Session()
m1 = Model(sess, "m1")
sess.run(tf.global_variables_initializer())
print('Learning Started!')
# train my model
for epoch in range(training_epochs):
avg_cost = 0
total_batch = int(mnist.train.num_examples / batch_size)
for i in range(total_batch):
batch_xs, batch_ys = mnist.train.next_batch(batch_size)
c, _ = m1.train(batch_xs, batch_ys)
avg_cost += c / total_batch
print('Epoch:', '%04d' % (epoch + 1), 'cost =', '{:.9f}'.format(avg_cost))
print('Learning Finished!')
# Test model and check accuracy
print('Accuracy:', m1.get_accuracy(mnist.test.images, mnist.test.labels))
[ # lab-11-5-mnist_cnn_ensemble_layers ]
# lab-11-5-mnist_cnn_ensemble_layers
" 각각 예측시키고, 후에 병합을 해서 예측치를 출력시킴"
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
# Lab 11 MNIST and Deep learning CNN
# https://www.tensorflow.org/tutorials/layers
import tensorflow as tf
import numpy as np
from tensorflow.examples.tutorials.mnist import input_data
tf.set_random_seed(777) # reproducibility
mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)
# Check out https://www.tensorflow.org/get_started/mnist/beginners for
# more information about the mnist dataset
# hyper parameters
learning_rate = 0.001
training_epochs = 20
batch_size = 100
class Model:
def __init__(self, sess, name):
self.sess = sess
self.name = name
self._build_net()
def _build_net(self):
with tf.variable_scope(self.name):
# dropout (keep_prob) rate 0.7~0.5 on training, but should be 1
# for testing
self.training = tf.placeholder(tf.bool)
# input place holders
self.X = tf.placeholder(tf.float32, [None, 784])
# img 28x28x1 (black/white), Input Layer
X_img = tf.reshape(self.X, [-1, 28, 28, 1])
self.Y = tf.placeholder(tf.float32, [None, 10])
# Convolutional Layer #1
conv1 = tf.layers.conv2d(inputs=X_img, filters=32, kernel_size=[3, 3],
padding="SAME", activation=tf.nn.relu)
# Pooling Layer #1
pool1 = tf.layers.max_pooling2d(inputs=conv1, pool_size=[2, 2],
padding="SAME", strides=2)
dropout1 = tf.layers.dropout(inputs=pool1,
rate=0.7, training=self.training)
# Convolutional Layer #2 and Pooling Layer #2
conv2 = tf.layers.conv2d(inputs=dropout1, filters=64, kernel_size=[3, 3],
padding="SAME", activation=tf.nn.relu)
pool2 = tf.layers.max_pooling2d(inputs=conv2, pool_size=[2, 2],
padding="SAME", strides=2)
dropout2 = tf.layers.dropout(inputs=pool2,
rate=0.7, training=self.training)
# Convolutional Layer #3 and Pooling Layer #3
conv3 = tf.layers.conv2d(inputs=dropout2, filters=128, kernel_size=[3, 3],
padding="SAME", activation=tf.nn.relu)
pool3 = tf.layers.max_pooling2d(inputs=conv3, pool_size=[2, 2],
padding="SAME", strides=2)
dropout3 = tf.layers.dropout(inputs=pool3,
rate=0.7, training=self.training)
# Dense Layer with Relu
flat = tf.reshape(dropout3, [-1, 128 * 4 * 4])
dense4 = tf.layers.dense(inputs=flat,
units=625, activation=tf.nn.relu)
dropout4 = tf.layers.dropout(inputs=dense4,
rate=0.5, training=self.training)
# Logits (no activation) Layer: L5 Final FC 625 inputs -> 10 outputs
self.logits = tf.layers.dense(inputs=dropout4, units=10)
# define cost/loss & optimizer
self.cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(
logits=self.logits, labels=self.Y))
self.optimizer = tf.train.AdamOptimizer(
learning_rate=learning_rate).minimize(self.cost)
correct_prediction = tf.equal(
tf.argmax(self.logits, 1), tf.argmax(self.Y, 1))
self.accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
def predict(self, x_test, training=False):
return self.sess.run(self.logits,
feed_dict={self.X: x_test, self.training: training})
def get_accuracy(self, x_test, y_test, training=False):
return self.sess.run(self.accuracy,
feed_dict={self.X: x_test,
self.Y: y_test, self.training: training})
def train(self, x_data, y_data, training=True):
return self.sess.run([self.cost, self.optimizer], feed_dict={
self.X: x_data, self.Y: y_data, self.training: training})
# initialize
sess = tf.Session()
models = []
num_models = 2
for m in range(num_models):
models.append(Model(sess, "model" + str(m)))
sess.run(tf.global_variables_initializer())
print('Learning Started!')
# train my model
for epoch in range(training_epochs):
avg_cost_list = np.zeros(len(models))
total_batch = int(mnist.train.num_examples / batch_size)
for i in range(total_batch):
batch_xs, batch_ys = mnist.train.next_batch(batch_size)
# train each model
for m_idx, m in enumerate(models):
c, _ = m.train(batch_xs, batch_ys)
avg_cost_list[m_idx] += c / total_batch
print('Epoch:', '%04d' % (epoch + 1), 'cost =', avg_cost_list)
print('Learning Finished!')
# Test model and check accuracy
test_size = len(mnist.test.labels)
predictions = np.zeros(test_size * 10).reshape(test_size, 10)
for m_idx, m in enumerate(models):
print(m_idx, 'Accuracy:', m.get_accuracy(
mnist.test.images, mnist.test.labels))
p = m.predict(mnist.test.images)
predictions += p
ensemble_correct_prediction = tf.equal(
tf.argmax(predictions, 1), tf.argmax(mnist.test.labels, 1))
ensemble_accuracy = tf.reduce_mean(
tf.cast(ensemble_correct_prediction, tf.float32))
print('Ensemble accuracy:', sess.run(ensemble_accuracy))
'''
0 Accuracy: 0.9933
1 Accuracy: 0.9946
2 Accuracy: 0.9934
3 Accuracy: 0.9935
4 Accuracy: 0.9935
5 Accuracy: 0.9949
6 Accuracy: 0.9941
Ensemble accuracy: 0.9952
'''
[ 참고자료 ]
https://www.inflearn.com/course/기본적인-머신러닝-딥러닝-강좌
https://github.com/hunkim/deeplearningzerotoall
https://www.tensorflow.org/api_docs/python/tf/layers
'프로젝트 > 인공지능' 카테고리의 다른 글
| [인공지능 #12 ] Q-Learning / OpenAI gym / frozenlake (0) | 2017.08.06 |
|---|---|
| [인공지능 #11 ] hello-rnn /char-seq-rnn /char-seq-softmax-only /rnn_long_char (0) | 2017.08.06 |
| [인공지능 #9]mnist_softmax /mnist_nn/mnist_nn_xavier/mnist_nn_deep / mnist_nn_dropout (0) | 2017.08.06 |
| [인공지능 #8]xor / xor-nn / xor-nn-wide-deep (0) | 2017.08.06 |
| [인공지능 #7] Rate overfiting , training/test data , nomalization (0) | 2017.08.05 |
[인공지능 #9]mnist_softmax /mnist_nn/mnist_nn_xavier/mnist_nn_deep / mnist_nn_dropout
인공지능 구현에 대한 글입니다.
글의 순서는 아래와 같습니다.
================================================
1. # lab-10-1-mnist_softmax
==># 정확도 90%
2. # lab-10-2-mnist_nn
3 단 깊이 /relu 적용=> 정확도 94 %
4. #lab-10-4-mnist_nn_deep
더 넓게(512), 더깊게 (L5) ==> 정확도 향상효과 없음
원인: OVERFITTING 대책 : Drop out 적용
7. 참고자료
=================================================
[ lab-10-1-mnist_softmax]
# lab-10-1-mnist_softmax
# 정확도 90%
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
# Lab 7 Learning rate and Evaluation
import tensorflow as tf
import random
#import matplotlib.pyplot as plt
from tensorflow.examples.tutorials.mnist import input_data
tf.set_random_seed(777) # reproducibility
mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)
# Check out https://www.tensorflow.org/get_started/mnist/beginners for
# more information about the mnist dataset
# parameters
learning_rate = 0.001
training_epochs = 15
batch_size = 100
# input place holders
X = tf.placeholder(tf.float32, [None, 784])
Y = tf.placeholder(tf.float32, [None, 10])
# weights & bias for nn layers
W = tf.Variable(tf.random_normal([784, 10]))
b = tf.Variable(tf.random_normal([10]))
hypothesis = tf.matmul(X, W) + b
# define cost/loss & optimizer
cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(
logits=hypothesis, labels=Y))
optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(cost)
# initialize
sess = tf.Session()
sess.run(tf.global_variables_initializer())
# train my model
for epoch in range(training_epochs):
avg_cost = 0
total_batch = int(mnist.train.num_examples / batch_size)
for i in range(total_batch):
batch_xs, batch_ys = mnist.train.next_batch(batch_size)
feed_dict = {X: batch_xs, Y: batch_ys}
c, _ = sess.run([cost, optimizer], feed_dict=feed_dict)
avg_cost += c / total_batch
print('Epoch:', '%04d' % (epoch + 1), 'cost =', '{:.9f}'.format(avg_cost))
print('Learning Finished!')
# Test model and check accuracy
correct_prediction = tf.equal(tf.argmax(hypothesis, 1), tf.argmax(Y, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
print('Accuracy:', sess.run(accuracy, feed_dict={
X: mnist.test.images, Y: mnist.test.labels}))
# Get one and predict
r = random.randint(0, mnist.test.num_examples - 1)
print("Label: ", sess.run(tf.argmax(mnist.test.labels[r:r + 1], 1)))
print("Prediction: ", sess.run(
tf.argmax(hypothesis, 1), feed_dict={X: mnist.test.images[r:r + 1]}))
#plt.imshow(mnist.test.images[r:r + 1].
# reshape(28, 28), cmap='Greys', interpolation='nearest')
#plt.show()
'''
Epoch: 0001 cost = 5.888845987
Epoch: 0002 cost = 1.860620173
Epoch: 0003 cost = 1.159035648
Epoch: 0004 cost = 0.892340870
Epoch: 0005 cost = 0.751155428
Epoch: 0006 cost = 0.662484806
Epoch: 0007 cost = 0.601544010
Epoch: 0008 cost = 0.556526115
Epoch: 0009 cost = 0.521186961
Epoch: 0010 cost = 0.493068354
Epoch: 0011 cost = 0.469686249
Epoch: 0012 cost = 0.449967254
Epoch: 0013 cost = 0.433519321
Epoch: 0014 cost = 0.419000337
Epoch: 0015 cost = 0.406490815
Learning Finished!
Accuracy: 0.9035
'''
# lab-10-2-mnist_nn
# 3 단 깊이 /relu 적용=> 정확도 94 %
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
# Lab 10 MNIST and NN
import tensorflow as tf
import random
# import matplotlib.pyplot as plt
from tensorflow.examples.tutorials.mnist import input_data
tf.set_random_seed(777) # reproducibility
mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)
# Check out https://www.tensorflow.org/get_started/mnist/beginners for
# more information about the mnist dataset
# parameters
learning_rate = 0.001
training_epochs = 15
batch_size = 100
# input place holders
X = tf.placeholder(tf.float32, [None, 784])
Y = tf.placeholder(tf.float32, [None, 10])
# weights & bias for nn layers
W1 = tf.Variable(tf.random_normal([784, 256]))
b1 = tf.Variable(tf.random_normal([256]))
L1 = tf.nn.relu(tf.matmul(X, W1) + b1)
W2 = tf.Variable(tf.random_normal([256, 256]))
b2 = tf.Variable(tf.random_normal([256]))
L2 = tf.nn.relu(tf.matmul(L1, W2) + b2)
W3 = tf.Variable(tf.random_normal([256, 10]))
b3 = tf.Variable(tf.random_normal([10]))
hypothesis = tf.matmul(L2, W3) + b3
# define cost/loss & optimizer
cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(
logits=hypothesis, labels=Y))
optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(cost)
# initialize
sess = tf.Session()
sess.run(tf.global_variables_initializer())
# train my model
for epoch in range(training_epochs):
avg_cost = 0
total_batch = int(mnist.train.num_examples / batch_size)
for i in range(total_batch):
batch_xs, batch_ys = mnist.train.next_batch(batch_size)
feed_dict = {X: batch_xs, Y: batch_ys}
c, _ = sess.run([cost, optimizer], feed_dict=feed_dict)
avg_cost += c / total_batch
print('Epoch:', '%04d' % (epoch + 1), 'cost =', '{:.9f}'.format(avg_cost))
print('Learning Finished!')
# Test model and check accuracy
correct_prediction = tf.equal(tf.argmax(hypothesis, 1), tf.argmax(Y, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
print('Accuracy:', sess.run(accuracy, feed_dict={
X: mnist.test.images, Y: mnist.test.labels}))
# Get one and predict
r = random.randint(0, mnist.test.num_examples - 1)
print("Label: ", sess.run(tf.argmax(mnist.test.labels[r:r + 1], 1)))
print("Prediction: ", sess.run(
tf.argmax(hypothesis, 1), feed_dict={X: mnist.test.images[r:r + 1]}))
# plt.imshow(mnist.test.images[r:r + 1].
# reshape(28, 28), cmap='Greys', interpolation='nearest')
# plt.show()
'''
Epoch: 0001 cost = 141.207671860
Epoch: 0002 cost = 38.788445864
Epoch: 0003 cost = 23.977515479
Epoch: 0004 cost = 16.315132428
Epoch: 0005 cost = 11.702554882
Epoch: 0006 cost = 8.573139748
Epoch: 0007 cost = 6.370995680
Epoch: 0008 cost = 4.537178684
Epoch: 0009 cost = 3.216900532
Epoch: 0010 cost = 2.329708954
Epoch: 0011 cost = 1.715552875
Epoch: 0012 cost = 1.189857912
Epoch: 0013 cost = 0.820965160
Epoch: 0014 cost = 0.624131458
Epoch: 0015 cost = 0.454633765
Learning Finished!
Accuracy: 0.9455
'''
#lab-10-4-mnist_nn_deep
# 더 넓게(512), 더깊게 (L5) ==> 정확도 향상효과 없음
# 원인: OVERFITTING 대책 : Drop out 적용
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
# Lab 10 MNIST and Deep learning
import tensorflow as tf
import random
# import matplotlib.pyplot as plt
from tensorflow.examples.tutorials.mnist import input_data
tf.set_random_seed(777) # reproducibility
mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)
# Check out https://www.tensorflow.org/get_started/mnist/beginners for
# more information about the mnist dataset
# parameters
learning_rate = 0.001
training_epochs = 15
batch_size = 100
# input place holders
X = tf.placeholder(tf.float32, [None, 784])
Y = tf.placeholder(tf.float32, [None, 10])
# weights & bias for nn layers
# http://stackoverflow.com/questions/33640581/how-to-do-xavier-initialization-on-tensorflow
W1 = tf.get_variable("W1", shape=[784, 512],
initializer=tf.contrib.layers.xavier_initializer())
b1 = tf.Variable(tf.random_normal([512]))
L1 = tf.nn.relu(tf.matmul(X, W1) + b1)
W2 = tf.get_variable("W2", shape=[512, 512],
initializer=tf.contrib.layers.xavier_initializer())
b2 = tf.Variable(tf.random_normal([512]))
L2 = tf.nn.relu(tf.matmul(L1, W2) + b2)
W3 = tf.get_variable("W3", shape=[512, 512],
initializer=tf.contrib.layers.xavier_initializer())
b3 = tf.Variable(tf.random_normal([512]))
L3 = tf.nn.relu(tf.matmul(L2, W3) + b3)
W4 = tf.get_variable("W4", shape=[512, 512],
initializer=tf.contrib.layers.xavier_initializer())
b4 = tf.Variable(tf.random_normal([512]))
L4 = tf.nn.relu(tf.matmul(L3, W4) + b4)
W5 = tf.get_variable("W5", shape=[512, 10],
initializer=tf.contrib.layers.xavier_initializer())
b5 = tf.Variable(tf.random_normal([10]))
hypothesis = tf.matmul(L4, W5) + b5
# define cost/loss & optimizer
cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(
logits=hypothesis, labels=Y))
optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(cost)
# initialize
sess = tf.Session()
sess.run(tf.global_variables_initializer())
# train my model
for epoch in range(training_epochs):
avg_cost = 0
total_batch = int(mnist.train.num_examples / batch_size)
for i in range(total_batch):
batch_xs, batch_ys = mnist.train.next_batch(batch_size)
feed_dict = {X: batch_xs, Y: batch_ys}
c, _ = sess.run([cost, optimizer], feed_dict=feed_dict)
avg_cost += c / total_batch
print('Epoch:', '%04d' % (epoch + 1), 'cost =', '{:.9f}'.format(avg_cost))
print('Learning Finished!')
# Test model and check accuracy
correct_prediction = tf.equal(tf.argmax(hypothesis, 1), tf.argmax(Y, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
print('Accuracy:', sess.run(accuracy, feed_dict={
X: mnist.test.images, Y: mnist.test.labels}))
# Get one and predict
r = random.randint(0, mnist.test.num_examples - 1)
print("Label: ", sess.run(tf.argmax(mnist.test.labels[r:r + 1], 1)))
print("Prediction: ", sess.run(
tf.argmax(hypothesis, 1), feed_dict={X: mnist.test.images[r:r + 1]}))
# plt.imshow(mnist.test.images[r:r + 1].
# reshape(28, 28), cmap='Greys', interpolation='nearest')
# plt.show()
'''
Epoch: 0001 cost = 0.266061549
Epoch: 0002 cost = 0.080796588
Epoch: 0003 cost = 0.049075800
Epoch: 0004 cost = 0.034772298
Epoch: 0005 cost = 0.024780529
Epoch: 0006 cost = 0.017072763
Epoch: 0007 cost = 0.014031383
Epoch: 0008 cost = 0.013763446
Epoch: 0009 cost = 0.009164047
Epoch: 0010 cost = 0.008291388
Epoch: 0011 cost = 0.007319742
Epoch: 0012 cost = 0.006434021
Epoch: 0013 cost = 0.005684378
Epoch: 0014 cost = 0.004781207
Epoch: 0015 cost = 0.004342310
Learning Finished!
Accuracy: 0.9742
'''
https://www.inflearn.com/course/기본적인-머신러닝-딥러닝-강좌
https://github.com/hunkim/deeplearningzerotoall
'프로젝트 > 인공지능' 카테고리의 다른 글
| [인공지능 #11 ] hello-rnn /char-seq-rnn /char-seq-softmax-only /rnn_long_char (0) | 2017.08.06 |
|---|---|
| [인공지능 #10 ]mnist_cnn/mnist_deep_cnn/mnist_cnn_class/mnist_cnn_layers/mnist_cnn_ensemble_layers (0) | 2017.08.06 |
| [인공지능 #8]xor / xor-nn / xor-nn-wide-deep (0) | 2017.08.06 |
| [인공지능 #7] Rate overfiting , training/test data , nomalization (0) | 2017.08.05 |
| [python] python 문법정리 (0) | 2017.08.05 |
[인공지능 #8]xor / xor-nn / xor-nn-wide-deep
인공지능 구현에 대한 글입니다.
글의 순서는 아래와 같습니다.
================================================
1. # lab-09-1-xor
==># 코딩은 이상이 없으나, 정확도는 50% 수준밖에는 않됩니다.
2. # lab-09-2-xor-nn
==>정확도 50%를 개선하는 방법으로 neural network ( 신경망 )을 이용하는 방법을 구현 ==> 정확도 100%
==>layer 1 이 다른곳에 입력으로 들어감 : 신경망 구성
layer1은 출력 2개 로 구성한다 , 이유는 최종 가설의 입력(2개의 data)으로 들어가기 때문임
정확도 1.0으로 개선이 됨
3. # lab-09-3-xor-nn-wide-deep
==> layer1의 출력을 10개 넓게 펴주고, 최종 가설에는 출력을 1개로 설정함
layer1,2,3로 더 깊게 만들고, 최종 출력을 1개로 설정함
==> 즉 더 넓게, 깊게 신경망을 구성해서 정확도 1.0으로 개선됨, 정밀도 높아짐 (큰것은 더 크게, 작은것은 더 작게)
4. 코딩탐구 필요(추가)
==> lab-09-4-xor_tensorboard
lab-09-5-linear_back_prop
lab-09-6-multi-linear_back_prop
lab-09-7-sigmoid_back_prop
lab-09-x-xor-nn-back_prop
5. 참고자료
=================================================
[rate overfitting , regularization tips]
1. COST 값이 줄지않고, 늘어난다면 Learning rate을 더 작게 변경해주어야함
반대로 너무 작게 줄거나, 도중에 멈춘다면 learing rate을 좀더 크게 해주어야함
보통 0.01을 기준으로 늘리거나 줄이면서 조정해 나가면 된다.
2. x data 값이 차이가 큰 경우, cost가 잘 줄지않거나, 학습이 잘 일어나지 않음 , 이럴때는 NOMALIZATION 을 해주어야 함.
3. OVERFITTING
- 더많은 TRAINING DATA
- REDUCE FEATURES
- REGULARIZATION ==> 구별선이 구부러지지 않토록 , X값을 일반화 시킴
[lab-09-1-xor]
# lab-09-1-xor
# 코딩은 이상이 없으나, 정확도는 50% 수준밖에는 않됩니다.
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
# Lab 9 XOR
import tensorflow as tf
import numpy as np
tf.set_random_seed(777) # for reproducibility
learning_rate = 0.1
x_data = [[0, 0],
[0, 1],
[1, 0],
[1, 1]]
y_data = [[0],
[1],
[1],
[0]]
x_data = np.array(x_data, dtype=np.float32)
y_data = np.array(y_data, dtype=np.float32)
X = tf.placeholder(tf.float32, [None, 2])
Y = tf.placeholder(tf.float32, [None, 1])
W = tf.Variable(tf.random_normal([2, 1]), name='weight')
b = tf.Variable(tf.random_normal([1]), name='bias')
# Hypothesis using sigmoid: tf.div(1., 1. + tf.exp(tf.matmul(X, W)))
hypothesis = tf.sigmoid(tf.matmul(X, W) + b)
# cost/loss function
cost = -tf.reduce_mean(Y * tf.log(hypothesis) + (1 - Y) *
tf.log(1 - hypothesis))
train = tf.train.GradientDescentOptimizer(learning_rate=learning_rate).minimize(cost)
# Accuracy computation
# True if hypothesis>0.5 else False
predicted = tf.cast(hypothesis > 0.5, dtype=tf.float32)
accuracy = tf.reduce_mean(tf.cast(tf.equal(predicted, Y), dtype=tf.float32))
# Launch graph
with tf.Session() as sess:
# Initialize TensorFlow variables
sess.run(tf.global_variables_initializer())
for step in range(10001):
sess.run(train, feed_dict={X: x_data, Y: y_data})
if step % 100 == 0:
print(step, sess.run(cost, feed_dict={
X: x_data, Y: y_data}), sess.run(W))
# Accuracy report
h, c, a = sess.run([hypothesis, predicted, accuracy],
feed_dict={X: x_data, Y: y_data})
print("\nHypothesis: ", h, "\nCorrect: ", c, "\nAccuracy: ", a)
'''
Hypothesis: [[ 0.5]
[ 0.5]
[ 0.5]
[ 0.5]]
Correct: [[ 0.]
[ 0.]
[ 0.]
[ 0.]]
Accuracy: 0.5
'''
# lab-09-2-xor-nn
""" 정확도 50%를 개선하는 방법으로 neural network ( 신경망 )을 이용하는 방법을 구현하함"""
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
# Lab 9 XOR
import tensorflow as tf
import numpy as np
tf.set_random_seed(777) # for reproducibility
learning_rate = 0.1
x_data = [[0, 0],
[0, 1],
[1, 0],
[1, 1]]
y_data = [[0],
[1],
[1],
[0]]
x_data = np.array(x_data, dtype=np.float32)
y_data = np.array(y_data, dtype=np.float32)
X = tf.placeholder(tf.float32, [None, 2])
Y = tf.placeholder(tf.float32, [None, 1])
W1 = tf.Variable(tf.random_normal([2, 2]), name='weight1')
b1 = tf.Variable(tf.random_normal([2]), name='bias1')
layer1 = tf.sigmoid(tf.matmul(X, W1) + b1)
""" layer 1 이 다른곳에 입력으로 연결함 : 신경망 구성
layer1은 입력1개, 출력 2로 구성한다 , 이유는 최종 가설의 입력으로 들어가기 때문임
정확도 1.0으로 개선이 됨
"""
W2 = tf.Variable(tf.random_normal([2, 1]), name='weight2')
b2 = tf.Variable(tf.random_normal([1]), name='bias2')
hypothesis = tf.sigmoid(tf.matmul(layer1, W2) + b2)
# cost/loss function
cost = -tf.reduce_mean(Y * tf.log(hypothesis) + (1 - Y) *
tf.log(1 - hypothesis))
train = tf.train.GradientDescentOptimizer(learning_rate=learning_rate).minimize(cost)
# Accuracy computation
# True if hypothesis>0.5 else False
predicted = tf.cast(hypothesis > 0.5, dtype=tf.float32)
accuracy = tf.reduce_mean(tf.cast(tf.equal(predicted, Y), dtype=tf.float32))
# Launch graph
with tf.Session() as sess:
# Initialize TensorFlow variables
sess.run(tf.global_variables_initializer())
for step in range(10001):
sess.run(train, feed_dict={X: x_data, Y: y_data})
if step % 100 == 0:
print(step, sess.run(cost, feed_dict={
X: x_data, Y: y_data}), sess.run([W1, W2]))
# Accuracy report
h, c, a = sess.run([hypothesis, predicted, accuracy],
feed_dict={X: x_data, Y: y_data})
print("\nHypothesis: ", h, "\nCorrect: ", c, "\nAccuracy: ", a)
'''
Hypothesis: [[ 0.01338218]
[ 0.98166394]
[ 0.98809403]
[ 0.01135799]]
Correct: [[ 0.]
[ 1.]
[ 1.]
[ 0.]]
Accuracy: 1.0
'''
# lab-09-3-xor-nn-wide-deep
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
# Lab 9 XOR
import tensorflow as tf
import numpy as np
tf.set_random_seed(777) # for reproducibility
learning_rate = 0.1
x_data = [[0, 0],
[0, 1],
[1, 0],
[1, 1]]
y_data = [[0],
[1],
[1],
[0]]
x_data = np.array(x_data, dtype=np.float32)
y_data = np.array(y_data, dtype=np.float32)
X = tf.placeholder(tf.float32, [None, 2])
Y = tf.placeholder(tf.float32, [None, 1])
W1 = tf.Variable(tf.random_normal([2, 10]), name='weight1')
b1 = tf.Variable(tf.random_normal([10]), name='bias1')
layer1 = tf.sigmoid(tf.matmul(X, W1) + b1)
""" layer1의 출력을 10개 넓게 펴주고, 최종 가설에는 출력을 1개로 설정함
layer1,2,3로 더 깊게 만들고, 최종 출력을 1개로 설정함
==> 즉 더 넓게, 깊게 신경망을 구성해서 정확도 1.0으로 개선됨 """
W2 = tf.Variable(tf.random_normal([10, 10]), name='weight2')
b2 = tf.Variable(tf.random_normal([10]), name='bias2')
layer2 = tf.sigmoid(tf.matmul(layer1, W2) + b2)
W3 = tf.Variable(tf.random_normal([10, 10]), name='weight3')
b3 = tf.Variable(tf.random_normal([10]), name='bias3')
layer3 = tf.sigmoid(tf.matmul(layer2, W3) + b3)
W4 = tf.Variable(tf.random_normal([10, 1]), name='weight4')
b4 = tf.Variable(tf.random_normal([1]), name='bias4')
hypothesis = tf.sigmoid(tf.matmul(layer3, W4) + b4)
# cost/loss function
cost = -tf.reduce_mean(Y * tf.log(hypothesis) + (1 - Y) *
tf.log(1 - hypothesis))
train = tf.train.GradientDescentOptimizer(learning_rate=learning_rate).minimize(cost)
# Accuracy computation
# True if hypothesis>0.5 else False
predicted = tf.cast(hypothesis > 0.5, dtype=tf.float32)
accuracy = tf.reduce_mean(tf.cast(tf.equal(predicted, Y), dtype=tf.float32))
# Launch graph
with tf.Session() as sess:
# Initialize TensorFlow variables
sess.run(tf.global_variables_initializer())
for step in range(10001):
sess.run(train, feed_dict={X: x_data, Y: y_data})
if step % 100 == 0:
print(step, sess.run(cost, feed_dict={
X: x_data, Y: y_data}), sess.run([W1, W2]))
# Accuracy report
h, c, a = sess.run([hypothesis, predicted, accuracy],
feed_dict={X: x_data, Y: y_data})
print("\nHypothesis: ", h, "\nCorrect: ", c, "\nAccuracy: ", a)
'''
Hypothesis: [[ 7.80511764e-04]
[ 9.99238133e-01]
[ 9.98379230e-01]
[ 1.55659032e-03]]
Correct: [[ 0.]
[ 1.]
[ 1.]
[ 0.]]
Accuracy: 1.0
'''
[참고자료 ]
https://www.inflearn.com/course/기본적인-머신러닝-딥러닝-강좌
https://github.com/hunkim/deeplearningzerotoall
'프로젝트 > 인공지능' 카테고리의 다른 글
| [인공지능 #10 ]mnist_cnn/mnist_deep_cnn/mnist_cnn_class/mnist_cnn_layers/mnist_cnn_ensemble_layers (0) | 2017.08.06 |
|---|---|
| [인공지능 #9]mnist_softmax /mnist_nn/mnist_nn_xavier/mnist_nn_deep / mnist_nn_dropout (0) | 2017.08.06 |
| [인공지능 #7] Rate overfiting , training/test data , nomalization (0) | 2017.08.05 |
| [python] python 문법정리 (0) | 2017.08.05 |
| [인공지능 #6 ] multinomial 적용 (0) | 2017.07.30 |
[인공지능 #7] Rate overfiting , training/test data , nomalization
인공지능 구현에 대한 글입니다.
글의 순서는 아래와 같습니다.
================================================
1. rate overfitting , regularization tips
2. train/test data sheet , learning rate , normalization(new)
- 학습과 테스트 data를 구분하는것이 합리적임
- mnist 소개
3. 참고자료
=================================================
[rate overfitting , regularization tips]
1. COST 값이 줄지않고, 늘어난다면 Learning rate을 더 작게 변경해주어야함
반대로 너무 작게 줄거나, 도중에 멈춘다면 learing rate을 좀더 크게 해주어야함
보통 0.01을 기준으로 늘리거나 줄이면서 조정해 나가면 된다.
2. x data 값이 차이가 큰 경우, cost가 잘 줄지않거나, 학습이 잘 일어나지 않음 , 이럴때는 NOMALIZATION 을 해주어야 함.
3. OVERFITTING
- 더많은 TRAINING DATA
- REDUCE FEATURES
- REGULARIZATION ==> 구별선이 구부러지지 않토록 , X값을 일반화 시킴
[train/test data sheet , learning rate , normalization(new)]
# <lab-07-1-learning_rate_and_evaluation>
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
# Lab 7 Learning rate and Evaluation
import tensorflow as tf
tf.set_random_seed(777) # for reproducibility
x_data = [[1, 2, 1],
[1, 3, 2],
[1, 3, 4],
[1, 5, 5],
[1, 7, 5],
[1, 2, 5],
[1, 6, 6],
[1, 7, 7]]
y_data = [[0, 0, 1],
[0, 0, 1],
[0, 0, 1],
[0, 1, 0],
[0, 1, 0],
[0, 1, 0],
[1, 0, 0],
[1, 0, 0]]
# Evaluation our model using this test dataset
x_test = [[2, 1, 1],
[3, 1, 2],
[3, 3, 4]]
y_test = [[0, 0, 1],
[0, 0, 1],
[0, 0, 1]]
X = tf.placeholder("float", [None, 3])
Y = tf.placeholder("float", [None, 3])
W = tf.Variable(tf.random_normal([3, 3]))
b = tf.Variable(tf.random_normal([3]))
# tf.nn.softmax computes softmax activations
# softmax = exp(logits) / reduce_sum(exp(logits), dim)
hypothesis = tf.nn.softmax(tf.matmul(X, W) + b)
# Cross entropy cost/loss
cost = tf.reduce_mean(-tf.reduce_sum(Y * tf.log(hypothesis), axis=1))
# Try to change learning_rate to small numbers
optimizer = tf.train.GradientDescentOptimizer(
learning_rate=0.1).minimize(cost)
# dhp Learning rate을 1.5로 움직이는 크기를 크게하면, H(y)값은 발산을 하게된다 : overfitting )
# Correct prediction Test model
prediction = tf.arg_max(hypothesis, 1)
is_correct = tf.equal(prediction, tf.arg_max(Y, 1))
accuracy = tf.reduce_mean(tf.cast(is_correct, tf.float32))
# Launch graph
with tf.Session() as sess:
# Initialize TensorFlow variables
sess.run(tf.global_variables_initializer())
for step in range(201):
cost_val, W_val, _ = sess.run(
[cost, W, optimizer], feed_dict={X: x_data, Y: y_data})
print(step, cost_val, W_val)
# 여기부터는 test data 로 test 진행함
# tf 입장에서는 처움보는 data (x)가 되는것이다.
# predict
print("Prediction:", sess.run(prediction, feed_dict={X: x_test}))
# Calculate the accuracy
print("Accuracy: ", sess.run(accuracy, feed_dict={X: x_test, Y: y_test}))
'''
when lr = 1.5
0 5.73203 [[-0.30548954 1.22985029 -0.66033536]
[-4.39069986 2.29670858 2.99386835]
[-3.34510708 2.09743214 -0.80419564]]
1 23.1494 [[ 0.06951046 0.29449689 -0.0999819 ]
[-1.95319986 -1.63627958 4.48935604]
[-0.90760708 -1.65020132 0.50593793]]
2 27.2798 [[ 0.44451016 0.85699677 -1.03748143]
[ 0.48429942 0.98872018 -0.57314301]
[ 1.52989244 1.16229868 -4.74406147]]
3 8.668 [[ 0.12396193 0.61504567 -0.47498202]
[ 0.22003263 -0.2470119 0.9268558 ]
[ 0.96035379 0.41933775 -3.43156195]]
4 5.77111 [[-0.9524312 1.13037777 0.08607888]
[-3.78651619 2.26245379 2.42393875]
[-3.07170963 3.14037919 -2.12054014]]
5 inf [[ nan nan nan]
[ nan nan nan]
[ nan nan nan]]
6 nan [[ nan nan nan]
[ nan nan nan]
[ nan nan nan]]
...
Prediction: [0 0 0]
Accuracy: 0.0
-------------------------------------------------
When lr = 1e-10
0 5.73203 [[ 0.80269563 0.67861295 -1.21728313]
[-0.3051686 -0.3032113 1.50825703]
[ 0.75722361 -0.7008909 -2.10820389]]
1 5.73203 [[ 0.80269563 0.67861295 -1.21728313]
[-0.3051686 -0.3032113 1.50825703]
[ 0.75722361 -0.7008909 -2.10820389]]
2 5.73203 [[ 0.80269563 0.67861295 -1.21728313]
[-0.3051686 -0.3032113 1.50825703]
[ 0.75722361 -0.7008909 -2.10820389]]
...
198 5.73203 [[ 0.80269563 0.67861295 -1.21728313]
[-0.3051686 -0.3032113 1.50825703]
[ 0.75722361 -0.7008909 -2.10820389]]
199 5.73203 [[ 0.80269563 0.67861295 -1.21728313]
[-0.3051686 -0.3032113 1.50825703]
[ 0.75722361 -0.7008909 -2.10820389]]
200 5.73203 [[ 0.80269563 0.67861295 -1.21728313]
[-0.3051686 -0.3032113 1.50825703]
[ 0.75722361 -0.7008909 -2.10820389]]
Prediction: [0 0 0]
Accuracy: 0.0
-------------------------------------------------
When lr = 0.1
0 5.73203 [[ 0.72881663 0.71536207 -1.18015325]
[-0.57753736 -0.12988332 1.60729778]
[ 0.48373488 -0.51433605 -2.02127004]]
1 3.318 [[ 0.66219079 0.74796319 -1.14612854]
[-0.81948912 0.03000021 1.68936598]
[ 0.23214608 -0.33772916 -1.94628811]]
2 2.0218 [[ 0.64342022 0.74127686 -1.12067163]
[-0.81161296 -0.00900121 1.72049117]
[ 0.2086665 -0.35079569 -1.909742 ]]
...
199 0.672261 [[-1.15377033 0.28146935 1.13632679]
[ 0.37484586 0.18958236 0.33544877]
[-0.35609841 -0.43973011 -1.25604188]]
200 0.670909 [[-1.15885413 0.28058422 1.14229572]
[ 0.37609792 0.19073224 0.33304682]
[-0.35536593 -0.44033223 -1.2561723 ]]
Prediction: [2 2 2]
Accuracy: 1.0
'''
# < lab-07-2-linear_regression_without_min_max >
# dhp data 편차가 터무니없이 크게되면, 학습이 잘 되질 않됨
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
import tensorflow as tf
import numpy as np
tf.set_random_seed(777) # for reproducibility
xy = np.array([[828.659973, 833.450012, 908100, 828.349976, 831.659973],
[823.02002, 828.070007, 1828100, 821.655029, 828.070007],
[819.929993, 824.400024, 1438100, 818.97998, 824.159973],
[816, 820.958984, 1008100, 815.48999, 819.23999],
[819.359985, 823, 1188100, 818.469971, 818.97998],
[819, 823, 1198100, 816, 820.450012],
[811.700012, 815.25, 1098100, 809.780029, 813.669983],
[809.51001, 816.659973, 1398100, 804.539978, 809.559998]])
x_data = xy[:, 0:-1]
y_data = xy[:, [-1]]
# placeholders for a tensor that will be always fed.
X = tf.placeholder(tf.float32, shape=[None, 4])
Y = tf.placeholder(tf.float32, shape=[None, 1])
W = tf.Variable(tf.random_normal([4, 1]), name='weight')
b = tf.Variable(tf.random_normal([1]), name='bias')
# Hypothesis
hypothesis = tf.matmul(X, W) + b
# Simplified cost/loss function
cost = tf.reduce_mean(tf.square(hypothesis - Y))
# Minimize
optimizer = tf.train.GradientDescentOptimizer(learning_rate=1e-5)
train = optimizer.minimize(cost)
# Launch the graph in a session.
sess = tf.Session()
# Initializes global variables in the graph.
sess.run(tf.global_variables_initializer())
for step in range(101):
cost_val, hy_val, _ = sess.run(
[cost, hypothesis, train], feed_dict={X: x_data, Y: y_data})
print(step, "Cost: ", cost_val, "\nPrediction:\n", hy_val)
'''
0 Cost: 2.45533e+12
Prediction:
[[-1104436.375]
[-2224342.75 ]
[-1749606.75 ]
[-1226179.375]
[-1445287.125]
[-1457459.5 ]
[-1335740.5 ]
[-1700924.625]]
1 Cost: 2.69762e+27
Prediction:
[[ 3.66371490e+13]
[ 7.37543360e+13]
[ 5.80198785e+13]
[ 4.06716290e+13]
[ 4.79336847e+13]
[ 4.83371348e+13]
[ 4.43026590e+13]
[ 5.64060907e+13]]
2 Cost: inf
Prediction:
[[ -1.21438790e+21]
[ -2.44468702e+21]
[ -1.92314724e+21]
[ -1.34811610e+21]
[ -1.58882674e+21]
[ -1.60219962e+21]
[ -1.46847142e+21]
[ -1.86965602e+21]]
3 Cost: inf
Prediction:
[[ 4.02525216e+28]
[ 8.10324465e+28]
[ 6.37453079e+28]
[ 4.46851237e+28]
[ 5.26638074e+28]
[ 5.31070676e+28]
[ 4.86744608e+28]
[ 6.19722623e+28]]
4 Cost: inf
Prediction:
[[ -1.33422428e+36]
[ -2.68593010e+36]
[ -2.11292430e+36]
[ -1.48114879e+36]
[ -1.74561303e+36]
[ -1.76030542e+36]
[ -1.61338091e+36]
[ -2.05415459e+36]]
5 Cost: inf
Prediction:
[[ inf]
[ inf]
[ inf]
[ inf]
[ inf]
[ inf]
[ inf]
[ inf]]
6 Cost: nan
Prediction:
[[ nan]
[ nan]
[ nan]
[ nan]
[ nan]
[ nan]
[ nan]
[ nan]]
'''
# <lab-07-3-linear_regression_min_max >
# dhp minmaxscala 함수를 이용해서, 정규화 (0~1사이의수)로 변환을 시킨다(nomallize)
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
import tensorflow as tf
import numpy as np
tf.set_random_seed(777) # for reproducibility
def MinMaxScaler(data):
numerator = data - np.min(data, 0)
denominator = np.max(data, 0) - np.min(data, 0)
# noise term prevents the zero division
return numerator / (denominator + 1e-7)
xy = np.array([[828.659973, 833.450012, 908100, 828.349976, 831.659973],
[823.02002, 828.070007, 1828100, 821.655029, 828.070007],
[819.929993, 824.400024, 1438100, 818.97998, 824.159973],
[816, 820.958984, 1008100, 815.48999, 819.23999],
[819.359985, 823, 1188100, 818.469971, 818.97998],
[819, 823, 1198100, 816, 820.450012],
[811.700012, 815.25, 1098100, 809.780029, 813.669983],
[809.51001, 816.659973, 1398100, 804.539978, 809.559998]])
# very important. It does not work without it.
xy = MinMaxScaler(xy)
print(xy)
x_data = xy[:, 0:-1]
y_data = xy[:, [-1]]
# placeholders for a tensor that will be always fed.
X = tf.placeholder(tf.float32, shape=[None, 4])
Y = tf.placeholder(tf.float32, shape=[None, 1])
W = tf.Variable(tf.random_normal([4, 1]), name='weight')
b = tf.Variable(tf.random_normal([1]), name='bias')
# Hypothesis
hypothesis = tf.matmul(X, W) + b
# Simplified cost/loss function
cost = tf.reduce_mean(tf.square(hypothesis - Y))
# Minimize
optimizer = tf.train.GradientDescentOptimizer(learning_rate=1e-5)
train = optimizer.minimize(cost)
# Launch the graph in a session.
sess = tf.Session()
# Initializes global variables in the graph.
sess.run(tf.global_variables_initializer())
for step in range(101):
cost_val, hy_val, _ = sess.run(
[cost, hypothesis, train], feed_dict={X: x_data, Y: y_data})
print(step, "Cost: ", cost_val, "\nPrediction:\n", hy_val)
'''
100 Cost: 0.152254
Prediction:
[[ 1.63450289]
[ 0.06628087]
[ 0.35014752]
[ 0.67070574]
# <lab-07-4-mnist_introduction>
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
# Lab 7 Learning rate and Evaluation
import tensorflow as tf
import random
# import matplotlib.pyplot as plt
tf.set_random_seed(777) # for reproducibility
from tensorflow.examples.tutorials.mnist import input_data
# Check out https://www.tensorflow.org/get_started/mnist/beginners for
# more information about the mnist dataset
mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)
nb_classes = 10
# MNIST data image of shape 28 * 28 = 784
X = tf.placeholder(tf.float32, [None, 784])
# 0 - 9 digits recognition = 10 classes
Y = tf.placeholder(tf.float32, [None, nb_classes])
W = tf.Variable(tf.random_normal([784, nb_classes]))
b = tf.Variable(tf.random_normal([nb_classes]))
# Hypothesis (using softmax)
hypothesis = tf.nn.softmax(tf.matmul(X, W) + b)
cost = tf.reduce_mean(-tf.reduce_sum(Y * tf.log(hypothesis), axis=1))
optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.1).minimize(cost)
# Test model
is_correct = tf.equal(tf.arg_max(hypothesis, 1), tf.arg_max(Y, 1))
# Calculate accuracy
accuracy = tf.reduce_mean(tf.cast(is_correct, tf.float32))
# parameters
training_epochs = 15
batch_size = 100
with tf.Session() as sess:
# Initialize TensorFlow variables
sess.run(tf.global_variables_initializer())
# Training cycle
for epoch in range(training_epochs):
avg_cost = 0
total_batch = int(mnist.train.num_examples / batch_size)
for i in range(total_batch):
batch_xs, batch_ys = mnist.train.next_batch(batch_size)
c, _ = sess.run([cost, optimizer], feed_dict={
X: batch_xs, Y: batch_ys})
avg_cost += c / total_batch
print('Epoch:', '%04d' % (epoch + 1),
'cost =', '{:.9f}'.format(avg_cost))
print("Learning finished")
# Test the model using test sets
print("Accuracy: ", accuracy.eval(session=sess, feed_dict={
X: mnist.test.images, Y: mnist.test.labels}))
# Get one and predict
r = random.randint(0, mnist.test.num_examples - 1)
print("Label: ", sess.run(tf.argmax(mnist.test.labels[r:r + 1], 1)))
print("Prediction: ", sess.run(
tf.argmax(hypothesis, 1), feed_dict={X: mnist.test.images[r:r + 1]}))
# 그래프 인쇄가 않됨, 향후 원인 파악필요함, 일단은 #로 주석처리 함
# don't know why this makes Travis Build error.
# plt.imshow(
# mnist.test.images[r:r + 1].reshape(28, 28),
# cmap='Greys',
# interpolation='nearest')
# plt.show()
'''
Epoch: 0001 cost = 2.868104637
Epoch: 0002 cost = 1.134684615
Epoch: 0003 cost = 0.908220728
Epoch: 0004 cost = 0.794199896
Epoch: 0005 cost = 0.721815854
Epoch: 0006 cost = 0.670184430
Epoch: 0007 cost = 0.630576546
Epoch: 0008 cost = 0.598888191
Epoch: 0009 cost = 0.573027079
Epoch: 0010 cost = 0.550497213
[참고자료 ]
'프로젝트 > 인공지능' 카테고리의 다른 글
| [인공지능 #9]mnist_softmax /mnist_nn/mnist_nn_xavier/mnist_nn_deep / mnist_nn_dropout (0) | 2017.08.06 |
|---|---|
| [인공지능 #8]xor / xor-nn / xor-nn-wide-deep (0) | 2017.08.06 |
| [python] python 문법정리 (0) | 2017.08.05 |
| [인공지능 #6 ] multinomial 적용 (0) | 2017.07.30 |
| [인공지능 #5 ] Logistics classification 가설함수 (0) | 2017.07.30 |
[python] python 문법정리
python 문법에 관한 글 입니다.
인공지능 tensor flow 코딩을 위해 python 문법에 대해 정리를 하고자 합니다.
글의 순서는 아래와 같습니다
==========================================
1. cheatsheet
- python문법의 컨닝 페이퍼입니다.
2. 주제별 코딩예제 모음
3. 참고한 자료
==========================================
[cheatsheet]
cheatsheet.pdf
[주제별 코딩예제 모음]
auth.pyprint('---수와 계산------------------------------------------------------------------------')
print(10 + 5)
print(10 - 5)
print(10 * 5)
print(10 / 5)
import math
print(math.ceil(2.2))
print(math.floor(2.7))
print(math.pow(2,10))
print(math.pi)
print('---문자와 데이터 타입----------------------------------------------------------------')
print('Hello')
print("Hello")
print("Hello 'world'")
print('Hello "world"')
print('Hello '+'world')
print('Hello '*3)
print('Hello'[0])
print('Hello'[1])
print('Hello'[2])
print('hello world'.capitalize())
print('hello world'.upper())
print('hello world'.__len__())
print(len('hello world'))
print('Hello world'.replace('world', 'programming'))
print("egoing's \"tutorial\"")
print("\\")
print("Hello\nworld")
print("Hello\t\tworld")
print("\a")
print('Hello\nworld')
print(10+5)
print("10"+"5")
print('---변수---------------------------------------------------------------------------')
x = 10
y = 5
print(x + y)
title = "python & ruby"
print("Title is " + title)
print('---문자열에서 변수의 사용---------------------------------------------------------------------------')
name = "이상효"
print("안녕하세요. "+name+"님")
print(name+"님을 위한 강의를 준비했습니다.")
print(name+"님 꼭 참석 부탁드립니다.")
print('---수계산 에서 변수의 사용---------------------------------------------------------------------------')
donation = 200
student = 10
sponsor = 100
print((donation*student)/sponsor)
print('---비교와 불리언---------------------------------------------------------------------------')
a=1
b=1
print(a==b)
print(1==2)
print(1>2)
print(1<2)
print(True)
print(False)
print('---조건문---------------------------------------------------------------------------')
if True:
print("code1")
print("code2")
print("code3")
print('---조건문의 활용---------------------------------------------------------------------------')
input = 11
real = 11
if real == input:
print("Hello!")
input = 11
real = 21
if real == input:
print("Hello!")
else:
print("Who are you?")
input = 33
real_egoing = 11
real_k8805 = "ab"
if real_egoing == input:
print("Hello!, egoing")
elif real_k8805 == input:
print("Hello!, k8805")
else:
print("Who are you?")
print('---입력과 출력---------------------------------------------------------------------------')
in_str = input("입력 해주세요.\n")
print(in_str.upper()+" World!")
print('---로그인 애플리케이션에 입력기능 추가하기-------------------------------------------------------------------------')
in_str = input("1 아이디를 입력해주세요.\n")
real_egoing = "11"
real_k8805 = "ab"
if real_egoing == in_str:
print("Hello!, egoing")
elif real_k8805 == in_str:
print("Hello!, k8805")
else:
print("Who are you?")
print('---논리 연산---------------------------------------------------------------------------')
in_str = input("2 아이디를 입력해주세요.\n")
real_egoing = "egoing"
real_k8805 = "k8805"
if real_egoing == in_str or real_k8805 == in_str:
print("Hello!")
else:
print("Who are you?")
input_id = input("아이디를 입력해주세요.\n")
input_pwd = input("비밀번호를 입력해주세요.\n")
real_id = "egoing"
real_pwd = "11"
if real_id == input_id:
if real_pwd == input_pwd:
print("Hello!")
else:
print("잘못된 비밀번호입니다")
else:
print("잘못된 아이디입니다")
input_id = input("아이디를 입력해주세요.\n")
input_pwd = input("비밀번호를 입력해주세요.\n")
real_id = "egoing"
real_pwd = "11"
if real_id == input_id and real_pwd == input_pwd:
print("Hello!")
else:
print("로그인에 실패했습니다")
print('---주석---------------------------------------------------------------------------')
'''
조건문 예제
egoing
2015
'''
# user input password
input = 33
real_egoing = 11
#real_k8805 = "ab"
if real_egoing == input:
print("Hello!, egoing")
#elif real_k8805 == input:
# print("Hello!, k8805")
else:
print("Who are you?")
print('---컨테이너---------------------------------------------------------------------------')
print(type('egoing')) # <class 'str'>
name = 'egoing'
print(name) # egoing
print(type(['egoing', 'leezche', 'graphittie'])) # <class 'list'>
names = ['egoing', 'leezche', 'graphittie']
print(names)
print(names[2]) # graphittie
egoing = ['programmer', 'seoul', 25, False]
egoing[1] = 'busan'
print(egoing) # ['programmer', 'busan', 25, False]
print('---사용설명서---------------------------------------------------------------------------')
al = ['A', 'B', 'C', 'D']
print(len(al)) # 4
al.append('E')
print(al) # ['A', 'B', 'C', 'D', 'E']
del (al[0])
print(al) # ['B', 'C', 'D', 'E']
print('---반복문---------------------------------------------------------------------------')
while False:
print('Hello world')
print('After while')
i = 0
while i < 3:
print('Hello world')
i = i + 1
i = 0
while i < 10:
print('print("Hello world ' + str(i * 9) + '")')
i = i + 1
i = 0
while i < 10:
if i == 4:
print(i)
i = i + 1
print('---컨테이너와 반복문-------------------------------------------------------------------------')
members = ['egoing', 'leezche', 'graphittie']
i = 0
while i < len(members):
print(members[i])
i = i + 1
members = ['egoing', 'leezche', 'graphittie']
for member in members:
print(member)
for item in range(5, 11):
print(item)
input_id = input("아이디를 입력해주세요.\n")
members = ['egoing', 'k8805', 'leezche']
for member in members:
if member == input_id:
print('Hello!, ' + member)
import sys
sys.exit()
print('Who are you?')
print('---함수---------------------------------------------------------------------------')
def a3():
print('aaa')
a3()
print('------------------------------------------------------------------------------------------')
def a3():
return 'aaa'
print(a3())
print('------------------------------------------------------------------------------------------')
def a(num):
return 'a' * num
print(a(3))
print('------------------------------------------------------------------------------------------')
input_id = input("아이디를 입력해주세요.\n")
def login(_id):
members = ['egoing', 'k8805', 'leezche']
for member in members:
if member == _id:
return True
return False
if login(input_id):
print('Hello, ' + input_id)
else:
print('Who are you?')
print('---모듈-------------------------------------------------------------------------------')
import math
print(math.ceil(2.9))
print(math.floor(2.9))
print(math.sqrt(16))
print('----모듈이 없을때-----------------------------------------------------------------------------------')
def a():
return 'B'
from egoing import a as z
import k8805 as k
print(z())
print(k.a())
print('---객체제작_객체사용---------------------------------------------------------------------------')
class Cal(object):
def __init__(self, v1, v2):
self.v1 = v1
self.v2 = v2
def add(self):
return self.v1 + self.v2
def subtract(self):
return self.v1 - self.v2
c1 = Cal(10, 10)
print(c1.add())
print(c1.subtract())
c2 = Cal(30, 20)
print(c2.add())
print(c2.subtract())
print('------객체제작_클래스-----------------------------------------------------------------')
class Cal(object):
c1 = Cal(10, 10)
print(c1.add())
print(c1.subtract())
c2 = Cal(30, 20)
print(c2.add())
print(c2.subtract())
print('------객체제작_생성자-----------------------------------------------------------------')
class Cal(object):
def __init__(self, v1, v2):
print(v1, v2)
c1 = Cal(10, 10)
# print(c1.add())
# print(c1.subtract())
c2 = Cal(30, 20)
# print(c2.add())
# print(c2.subtract())
print('------객체제작_인스턴스 변수와 메소드-----------------------------------------------------------------')
class Cal(object):
def __init__(self, v1, v2):
self.v1 = v1
self.v2 = v2
def add(self):
return self.v1 + self.v2
def subtract(self):
return self.v1 - self.v2
c1 = Cal(10, 10)
print(c1.add())
print(c1.subtract())
c2 = Cal(30, 20)
print(c2.add())
print(c2.subtract())
print('------객체와변수_인스턴스 변수의 특성-----------------------------------------------------------------')
class C(object):
def __init__(self, v):
self.value = v
def show(self):
print(self.value)
c1 = C(10)
print(c1.value)
c1.value = 20
print(c1.value)
c1.show()
print('------객체와변수_set/get 메소드--------------------------------------------------------------')
class C(object):
def __init__(self, v):
self.value = v
def show(self):
print(self.value)
def getValue(self):
return self.value
def setValue(self, v):
self.value = v
c1 = C(10)
print(c1.getValue())
c1.setValue(20)
print(c1.getValue())
print('------객체와변수_set/get 메소드를 사용해야하는 이유--------------------------------------------------')
class Cal(object):
def __init__(self, v1, v2):
if isinstance(v1, int):
self.v1 = v1
if isinstance(v2, int):
self.v2 = v2
def add(self):
return self.v1 + self.v2
def subtract(self):
return self.v1 - self.v2
def setV1(self, v):
if isinstance(v, int): # v 가 int class의 인스턴스인지 확인
self.v1 = v
def getV1(self):
return self.v1
c1 = Cal(10, 10)
print(c1.add())
print(c1.subtract())
c1.setV1('one')
c1.v2 = 30
print(c1.add())
print(c1.subtract())
print('---------객체와변수_속성 : 실행않됨---------------------------------------------------------------------')
"""
class C(object):
def __init__(self, v):
self.__value = v
def show(self):
print(self.__value)
c1 = C(10)
print(c1.__value)
c1.show()
"""
print('---상속_상속의 문법---------------------------------------------------------------------------')
class Class1(object):
def method1(self): return 'm1'
c1 = Class1()
print(c1, c1.method1())
class Class3(Class1):
def method2(self): return 'm2'
c3 = Class3()
print(c3, c3.method1())
print(c3, c3.method2())
class Class2(object):
def method1(self): return 'm1'
def method2(self): return 'm2'
c2 = Class2()
print(c2, c2.method1())
print(c2, c2.method2())
print('-----상속_상속의 응용-------------------------------------------------------------------')
"""
클래스의 재활용/함수추가 : 타인의 또는 자신의 클래스를 상속하고, 새로운 기능추가, 상속은 변수까지도 이어받음
"""
class Cal(object):
def __init__(self, v1, v2):
if isinstance(v1, int):
self.v1 = v1
if isinstance(v2, int):
self.v2 = v2
def add(self):
return self.v1 + self.v2
def subtract(self):
return self.v1 - self.v2
def setV1(self, v):
if isinstance(v, int):
self.v1 = v
def getV1(self):
return self.v1
class CalMultiply(Cal):
def multiply(self):
return self.v1 * self.v2
class CalDivide(CalMultiply):
def divide(self):
return self.v1 / self.v2
c1 = CalMultiply(10, 10)
print(c1.add())
print(c1.multiply())
c2 = CalDivide(20, 10)
print(c2, c2.add())
print(c2, c2.multiply())
print(c2, c2.divide())
print('-------------clss멤버_클래스 매소드--------------------------------------------------------------')
class Cs:
@staticmethod
def static_method():
print("Static method")
@classmethod
def class_method(cls):
print("Class method")
def instance_method(self):
print("Instance method")
i = Cs()
Cs.static_method()
Cs.class_method()
i.instance_method()
print('-------------clss멤버_클래스 변수---------------------------------------------------------------')
class Cs:
count = 0
def __init__(self):
Cs.count = Cs.count + 1
@classmethod
def getCount(cls):
return Cs.count
i1 = Cs()
i2 = Cs()
i3 = Cs()
i4 = Cs()
print(Cs.getCount())
print('-------------clss멤버_clss member의 활용---------------------------------------------------------------')
class Cal(object):
_history = []
def __init__(self, v1, v2):
if isinstance(v1, int):
self.v1 = v1
if isinstance(v2, int):
self.v2 = v2
def add(self):
result = self.v1 + self.v2
Cal._history.append("add : %d+%d=%d" % (self.v1, self.v2, result))
return result
def subtract(self):
result = self.v1 - self.v2
Cal._history.append("subtract : %d-%d=%d" % (self.v1, self.v2, result))
return result
def setV1(self, v):
if isinstance(v, int):
self.v1 = v
def getV1(self):
return self.v1
@classmethod
def history(cls):
for item in Cal._history:
print(item)
class CalMultiply(Cal):
def multiply(self):
result = self.v1 * self.v2
Cal._history.append("multiply : %d*%d=%d" % (self.v1, self.v2, result))
return result
class CalDivide(CalMultiply):
def divide(self):
result = self.v1 / self.v2
Cal._history.append("divide : %d/%d=%d" % (self.v1, self.v2, result))
return result
c1 = CalMultiply(10, 10)
print(c1.add())
print(c1.multiply())
c2 = CalDivide(20, 10)
print(c2, c2.add())
print(c2, c2.multiply())
print(c2, c2.divide())
Cal.history()
print('-------------override_형식---------------------------------------------------------------')
class C1:
def m(self):
return 'parent'
class C2(C1):
def m(self):
return super().m() + ' child'
pass
o = C2()
print(o.m())
print('-------------override_override 활용---------------------------------------------------------------')
class Cal(object):
_history = []
def __init__(self, v1, v2):
if isinstance(v1, int):
self.v1 = v1
if isinstance(v2, int):
self.v2 = v2
def add(self):
result = self.v1 + self.v2
Cal._history.append("add : %d+%d=%d" % (self.v1, self.v2, result))
return result
def subtract(self):
result = self.v1 - self.v2
Cal._history.append("subtract : %d-%d=%d" % (self.v1, self.v2, result))
return result
def setV1(self, v):
if isinstance(v, int):
self.v1 = v
def getV1(self):
return self.v1
@classmethod
def history(cls):
for item in Cal._history:
print(item)
def info(self):
return "Cal => v1 : %d, v2 : %d" % (self.v1, self.v2)
class CalMultiply(Cal):
def multiply(self):
result = self.v1 * self.v2
Cal._history.append("multiply : %d*%d=%d" % (self.v1, self.v2, result))
return result
def info(self):
return "CalMultiply => %s" % super().info()
class CalDivide(CalMultiply):
def divide(self):
result = self.v1 / self.v2
Cal._history.append("divide : %d/%d=%d" % (self.v1, self.v2, result))
return result
def info(self):
return "CalDivide => %s" % super().info()
c0 = Cal(30, 60)
print(c0.info())
c1 = CalMultiply(10, 10)
print(c1.info())
c2 = CalDivide(20, 10)
print(c2.info())
print('-------------객체와 모듈---------------------------------------------------------------')
import lib
obj = lib.A()
print(obj.a())
print('-------------다중상속_형식,단점---------------------------------------------------------------')
class C1():
def c1_m(self):
print("c1_m")
def m(self):
print("C1 m")
class C2():
def c2_m(self):
print("c2_m")
def m(self):
print("C2 m")
class C3(C2, C1):
def m(self):
print("C3 m")
c = C3()
c.c1_m()
c.c2_m()
c.m()
print(C3.__mro__)
print('-------------다중상속_다중상속의 활용---------------------------------------------------------------')
class CalMultiply():
def multiply(self):
return self.v1 * self.v2
class CalDivide():
def divide(self):
return self.v1 / self.v2
class Cal(CalMultiply, CalDivide): # 왼쪽부모가 우선순위가 높다
def __init__(self, v1, v2):
if isinstance(v1, int):
self.v1 = v1
if isinstance(v2, int):
self.v2 = v2
def add(self):
return self.v1 + self.v2
def subtract(self):
return self.v1 - self.v2
def setV1(self, v):
if isinstance(v, int):
self.v1 = v
def getV1(self):
return self.v1
c = Cal(100, 10)
print(c.add())
print(c.multiply())
print(c.divide())
[참고한 자료]
'프로젝트 > 인공지능' 카테고리의 다른 글
| [인공지능 #8]xor / xor-nn / xor-nn-wide-deep (0) | 2017.08.06 |
|---|---|
| [인공지능 #7] Rate overfiting , training/test data , nomalization (0) | 2017.08.05 |
| [인공지능 #6 ] multinomial 적용 (0) | 2017.07.30 |
| [인공지능 #5 ] Logistics classification 가설함수 (0) | 2017.07.30 |
| [인공지능 #4 ] 여러개의 DATA를 (X 변수) 구현 (0) | 2017.07.30 |
[인공지능 #6 ] multinomial 적용
[ multinomial 적용]
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
# Lab 6 Softmax Classifier
import tensorflow as tf
tf.set_random_seed(777) # for reproducibility
x_data = [[1, 2, 1, 1],
[2, 1, 3, 2],
[3, 1, 3, 4],
[4, 1, 5, 5],
[1, 7, 5, 5],
[1, 2, 5, 6],
[1, 6, 6, 6],
[1, 7, 7, 7]]
y_data = [[0, 0, 1], ==>2
[0, 0, 1], ==>2
[0, 0, 1],
[0, 1, 0], ==>1
[0, 1, 0],
[0, 1, 0],
[1, 0, 0],
[1, 0, 0]]
X = tf.placeholder("float", [None, 4])
Y = tf.placeholder("float", [None, 3])
nb_classes = 3
W = tf.Variable(tf.random_normal([4, nb_classes]), name='weight')
b = tf.Variable(tf.random_normal([nb_classes]), name='bias')
# tf.nn.softmax computes softmax activations
==> 확율( 1~0 )으로 변환해줌
# softmax = exp(logits) / reduce_sum(exp(logits), dim)
hypothesis = tf.nn.softmax(tf.matmul(X, W) + b)
# Cross entropy cost/loss
cost = tf.reduce_mean(-tf.reduce_sum(Y * tf.log(hypothesis), axis=1))
D(S,L)=-SIGMA Li log(Si) 함수와
는 같은 결과를 나타냄
예측이 맞으면 COST는 작아지고, 틀리면 커지도록 구성한 함수임.
상기 함수(상기 2함수는 같은결과를 내고있음 , 같은 함수라고 할수있음)가 그 역할을 충실히 해내고 있음.
optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.1).minimize(cost)
COST 최소화 실행
# Launch graph
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
for step in range(2001):
sess.run(optimizer, feed_dict={X: x_data, Y: y_data})
if step % 200 == 0:
print(step, sess.run(cost, feed_dict={X: x_data, Y: y_data}))
print('--------------')
# Testing & One-hot encoding
a = sess.run(hypothesis, feed_dict={X: [[1, 11, 7, 9]]})
print(a, sess.run(tf.arg_max(a, 1)))
==> 예측치 출력(3개) , one hot 값 출력 ==> 어떤 클래스 인지를 결정해서 출력해줌
==> 예시)손글씨의 특징을 3개의 x 값으로 표현하고, 예측치를 출력 3개로 하며, 이때 확율이 가장높게나타난 클래스가 해당 손글씨가 됨.
==> 예시) 글래스(강아지,고양이...)등의 특징을 여러 x갋으로 표현하고, 예측을 y값으로 예측하고, 그중 확율이 제일 높은수치를 선택해줌, 제일 큰 수치가 해당클래스(고양이, 강아지 등)가 된다.
print('--------------')
b = sess.run(hypothesis, feed_dict={X: [[1, 3, 4, 3]]})
print(b, sess.run(tf.arg_max(b, 1)))
print('--------------')
c = sess.run(hypothesis, feed_dict={X: [[1, 1, 0, 1]]})
print(c, sess.run(tf.arg_max(c, 1)))
print('--------------')
all = sess.run(hypothesis, feed_dict={
X: [[1, 11, 7, 9], [1, 3, 4, 3], [1, 1, 0, 1]]})
print(all, sess.run(tf.arg_max(all, 1)))
'''
--------------
[[ 1.38904958e-03 9.98601854e-01 9.06129117e-06]] [1] ==> 전체의 합은 1 임
--------------
[[ 0.93119204 0.06290206 0.0059059 ]] [0]
--------------
[[ 1.27327668e-08 3.34112905e-04 9.99665856e-01]] [2]
--------------
[[ 1.38904958e-03 9.98601854e-01 9.06129117e-06]
[ 9.31192040e-01 6.29020557e-02 5.90589503e-03]
[ 1.27327668e-08 3.34112905e-04 9.99665856e-01]] [1 0 2]
'''
[ Fancy softmax classfication 구현 ]
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
# Lab 6 Softmax Classifier
import tensorflow as tf
import numpy as np
tf.set_random_seed(777) # for reproducibility
# Predicting animal type based on various features
xy = np.loadtxt('data-04-zoo.csv', delimiter=',', dtype=np.float32)
x_data = xy[:, 0:-1]
y_data = xy[:, [-1]]
print(x_data.shape, y_data.shape)
nb_classes = 7 # 0 ~ 6
X = tf.placeholder(tf.float32, [None, 16])
Y = tf.placeholder(tf.int32, [None, 1]) # 0 ~ 6
Y_one_hot = tf.one_hot(Y, nb_classes) # one hot
print("one_hot", Y_one_hot)
Y_one_hot = tf.reshape(Y_one_hot, [-1, nb_classes])
# one hot을 하면 rank가 1개 늘어나게 된다, 이를 다시 줄이는 역할을 하는것이 reshape 임
print("reshape", Y_one_hot)
W = tf.Variable(tf.random_normal([16, nb_classes]), name='weight')
b = tf.Variable(tf.random_normal([nb_classes]), name='bias')
# tf.nn.softmax computes softmax activations
# softmax = exp(logits) / reduce_sum(exp(logits), dim)
logits = tf.matmul(X, W) + b
hypothesis = tf.nn.softmax(logits)
# Cross entropy cost/loss
cost_i = tf.nn.softmax_cross_entropy_with_logits(logits=logits,
labels=Y_one_hot)
cost = tf.reduce_mean(cost_i)
optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.1).minimize(cost)
prediction = tf.argmax(hypothesis, 1)
correct_prediction = tf.equal(prediction, tf.argmax(Y_one_hot, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
# Launch graph
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
for step in range(2000):
sess.run(optimizer, feed_dict={X: x_data, Y: y_data})
if step % 100 == 0:
loss, acc = sess.run([cost, accuracy], feed_dict={
X: x_data, Y: y_data})
print("Step: {:5}\tLoss: {:.3f}\tAcc: {:.2%}".format(
step, loss, acc))
# Let's see if we can predict
pred = sess.run(prediction, feed_dict={X: x_data})
# y_data: (N,1) = flatten => (N, ) matches pred.shape
for p, y in zip(pred, y_data.flatten()):
print("[{}] Prediction: {} True Y: {}".format(p == int(y), p, int(y)))
'''
Step: 0 Loss: 5.106 Acc: 37.62%
Step: 100 Loss: 0.800 Acc: 79.21%
Step: 200 Loss: 0.486 Acc: 88.12%
Step: 300 Loss: 0.349 Acc: 90.10%
Step: 400 Loss: 0.272 Acc: 94.06%
Step: 500 Loss: 0.222 Acc: 95.05%
Step: 600 Loss: 0.187 Acc: 97.03%
Step: 700 Loss: 0.161 Acc: 97.03%
Step: 800 Loss: 0.140 Acc: 97.03%
Step: 900 Loss: 0.124 Acc: 97.03%
Step: 1000 Loss: 0.111 Acc: 97.03%
Step: 1100 Loss: 0.101 Acc: 99.01%
Step: 1200 Loss: 0.092 Acc: 100.00%
[참고자료 ]
https://www.inflearn.com/course/기본적인-머신러닝-딥러닝-강좌/
http://agiantmind.tistory.com/176
https://www.tensorflow.org/install/
https://github.com/hunkim/deeplearningzerotoall
http://www.derivative-calculator.net/
http://terms.naver.com/entry.nhn?docId=3350391&cid=58247&categoryId=58247 ==> 미분계산/공식
http://matplotlib.org/users/installing.html ==>matplotlib 설치
https://www.kaggle.com/==> DATA SAMPLE 을 구할수 있는 사이트
'프로젝트 > 인공지능' 카테고리의 다른 글
| [인공지능 #7] Rate overfiting , training/test data , nomalization (0) | 2017.08.05 |
|---|---|
| [python] python 문법정리 (0) | 2017.08.05 |
| [인공지능 #5 ] Logistics classification 가설함수 (0) | 2017.07.30 |
| [인공지능 #4 ] 여러개의 DATA를 (X 변수) 구현 (0) | 2017.07.30 |
| [인공지능 #3 ] Cost 최소화 알고리즘 구현 (0) | 2017.07.30 |
[인공지능 #5 ] Logistics classification 가설함수
python 문법에 관한 글 입니다.
인공지능 tensor flow 코딩을 위해 python 문법에 대해 정리를 하고자 합니다.
글의 순서는 아래와 같습니다
==========================================
1.Logistics classification 가설함수
==> 합격, 불합격을 구분할때 적용함
2. 당뇨병 예측
3. Next step
==> Multinomial : 여러개의 classification의 확율을 구함.
4. 참고자료
============================================
[ Logistics classification 가설함수]
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
# Lab 5 Logistic Regression Classifier
import tensorflow as tf
tf.set_random_seed(777) # for reproducibility
x_data = [[1, 2],
[2, 3],
[3, 1],
[4, 3],
[5, 3],
[6, 2]]
# x data에 따라 합부(0,1)로 나타는 Y data를 예측함
# 공부한 시간에 따른 합격여부 data
y_data = [[0],
[0],
[0],
[1],
[1],
[1]]
# placeholders for a tensor that will be always fed.
X = tf.placeholder(tf.float32, shape=[None, 2])
Y = tf.placeholder(tf.float32, shape=[None, 1])
W = tf.Variable(tf.random_normal([2, 1]), name='weight')
b = tf.Variable(tf.random_normal([1]), name='bias')
# sigmoid 함수 사용함
==> h(x) 가 0~1 사이에 값을 갖게됨.
==> cost(w)은 y값과 h(x)차이가 크면 1에 가깝고, 작으면 0에 가까워지도록 구현함 , 이를 하나의 공식으로 통일시킴.
-기존의 cost함수형식으로 하게되면 울퉁불퉁한 2차 곡선 형태되고, 경사를 내려오면서 수많은 기울기 0 지점을 지나게 되어, 목표지점에 다다르지 못하게됨
-따라서 다른 코스트 함수가 필요함, 아래 함수가 그 대안임
# Hypothesis using sigmoid: tf.div(1., 1. + tf.exp(tf.matmul(X, W)))
hypothesis = tf.sigmoid(tf.matmul(X, W) + b)
# cost/loss function
cost = -tf.reduce_mean(Y * tf.log(hypothesis) + (1 - Y) *
tf.log(1 - hypothesis))
train = tf.train.GradientDescentOptimizer(learning_rate=0.01).minimize(cost)
# Accuracy computation
# True if hypothesis>0.5 else False
predicted = tf.cast(hypothesis > 0.5, dtype=tf.float32)
accuracy = tf.reduce_mean(tf.cast(tf.equal(predicted, Y), dtype=tf.float32))
# Launch graph
with tf.Session() as sess:
# Initialize TensorFlow variables
sess.run(tf.global_variables_initializer())
for step in range(10001):
cost_val, _ = sess.run([cost, train], feed_dict={X: x_data, Y: y_data})
if step % 200 == 0:
print(step, cost_val)
# Accuracy report
h, c, a = sess.run([hypothesis, predicted, accuracy],
feed_dict={X: x_data, Y: y_data})
print("\nHypothesis: ", h, "\nCorrect (Y): ", c, "\nAccuracy: ", a)
'''
0 1.73078
200 0.571512
400 0.507414
600 0.471824
800 0.447585
...
9200 0.159066
9400 0.15656
9600 0.154132
9800 0.151778
10000 0.149496
Hypothesis: [[ 0.03074029]
[ 0.15884677]
[ 0.30486736]
[ 0.78138196]
[ 0.93957496]
[ 0.98016882]]
y 예측치
Correct (Y): [[ 0.]
[ 0.]
[ 0.]
[ 1.]
[ 1.]
[ 1.]]
y 예측치와 실제 y값의 차가 0.5 이상이면 0. 이하면 1
Accuracy: 1.0
예측정확도 100%
'''
'''
[당뇨병 예측]
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
# Lab 5 Logistic Regression Classifier
import tensorflow as tf
import numpy as np
tf.set_random_seed(777) # for reproducibility
xy = np.loadtxt('data-03-diabetes.csv', delimiter=',', dtype=np.float32)
x_data = xy[:, 0:-1]
y_data = xy[:, [-1]]
print(x_data.shape, y_data.shape)
# placeholders for a tensor that will be always fed.
X = tf.placeholder(tf.float32, shape=[None, 8])
Y = tf.placeholder(tf.float32, shape=[None, 1])
W = tf.Variable(tf.random_normal([8, 1]), name='weight')
b = tf.Variable(tf.random_normal([1]), name='bias')
# Hypothesis using sigmoid: tf.div(1., 1. + tf.exp(tf.matmul(X, W)))
hypothesis = tf.sigmoid(tf.matmul(X, W) + b)
# cost/loss function
cost = -tf.reduce_mean(Y * tf.log(hypothesis) + (1 - Y) *
tf.log(1 - hypothesis))
train = tf.train.GradientDescentOptimizer(learning_rate=0.01).minimize(cost)
# Accuracy computation
# True if hypothesis>0.5 else False
predicted = tf.cast(hypothesis > 0.5, dtype=tf.float32)
accuracy = tf.reduce_mean(tf.cast(tf.equal(predicted, Y), dtype=tf.float32))
# Launch graph
with tf.Session() as sess:
# Initialize TensorFlow variables
sess.run(tf.global_variables_initializer())
for step in range(10001):
cost_val, _ = sess.run([cost, train], feed_dict={X: x_data, Y: y_data})
if step % 200 == 0:
print(step, cost_val)
# Accuracy report
h, c, a = sess.run([hypothesis, predicted, accuracy],
feed_dict={X: x_data, Y: y_data})
print("\nHypothesis: ", h, "\nCorrect (Y): ", c, "\nAccuracy: ", a)
'''
0 0.82794
200 0.755181
400 0.726355
600 0.705179
800 0.686631
...
9600 0.492056
9800 0.491396
10000 0.490767
...
[ 1.]
[ 1.]
[ 1.]]
Accuracy: 0.762846
[ 참고자료 ]
https://www.inflearn.com/course/기본적인-머신러닝-딥러닝-강좌/
http://agiantmind.tistory.com/176
https://www.tensorflow.org/install/
https://github.com/hunkim/deeplearningzerotoall
http://www.derivative-calculator.net/
http://terms.naver.com/entry.nhn?docId=3350391&cid=58247&categoryId=58247 ==> 미분계산/공식
http://matplotlib.org/users/installing.html ==>matplotlib 설치
https://www.kaggle.com/==> DATA SAMPLE 을 구할수 있는 사이트
'''
'프로젝트 > 인공지능' 카테고리의 다른 글
| [python] python 문법정리 (0) | 2017.08.05 |
|---|---|
| [인공지능 #6 ] multinomial 적용 (0) | 2017.07.30 |
| [인공지능 #4 ] 여러개의 DATA를 (X 변수) 구현 (0) | 2017.07.30 |
| [인공지능 #3 ] Cost 최소화 알고리즘 구현 (0) | 2017.07.30 |
| [인공지능 #2 ] Tensorflow Linear regression 구현 (0) | 2017.07.30 |
[인공지능 #4 ] 여러개의 DATA를 (X 변수) 구현
인공지능 구현에 대한 글입니다.
글의 순서는 아래와 같습니다.
================================================
1. X의 갯수가3개인 경우
==> 코딩이 매우 복잡해짐 , 대안으로 "매트릭스_행렬 "을 사용하면 간단해짐.
2. 매트릭스 적용
==> DATA 갯수(X 변수의갯수) 이 많아져도 간단하게 코딩이 가능해짐
3. TENSOR FLOW로 파일에서 DATA 읽어오기
==> DATA를 프로그램상에 올리면 메모리한계가 있으므로, DATA를 별도의 화일에 저장하고 불러오는것이 유리함
4. DATA QUE로 읽어들임 시간차로 처리해서 시스템 부하를 줄어줌
==> 화일이 많거나, 용량이 커지면 TENSORFLOW에서 자동으로 시간차를 두고 처리해주는 시스템 구현이 가능함
5. Next step
==> LOGISTICS 알고리즘 : CLASSSIFICATION 중 주요한 알고리즘.
6.참고자료
=================================================
[ Cost 최소화 알고리즘 구현 소스코드 분석 ]
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
# Lab 4 Multi-variable linear regression
import tensorflow as tf
tf.set_random_seed(777) # for reproducibility
==> x값이 3개
x1_data = [73., 93., 89., 96., 73.]
x2_data = [80., 88., 91., 98., 66.]
x3_data = [75., 93., 90., 100., 70.]
y_data = [152., 185., 180., 196., 142.]
# placeholders for a tensor that will be always fed.
x1 = tf.placeholder(tf.float32)
x2 = tf.placeholder(tf.float32)
x3 = tf.placeholder(tf.float32)
Y = tf.placeholder(tf.float32)
w1 = tf.Variable(tf.random_normal([1]), name='weight1')
w2 = tf.Variable(tf.random_normal([1]), name='weight2')
w3 = tf.Variable(tf.random_normal([1]), name='weight3')
b = tf.Variable(tf.random_normal([1]), name='bias')
hypothesis = x1 * w1 + x2 * w2 + x3 * w3 + b
print(hypothesis)
==> x,w값이 3개(x1,x2,x3 ,w1,w2,w3) 인 가설임
==> 코딩이 매우 복잡해짐 , 대안으로 "매트릭스_행렬 "을 사용하면 간단해짐.
# cost/loss function
cost = tf.reduce_mean(tf.square(hypothesis - Y))
# Minimize. Need a very small learning rate for this data set
optimizer = tf.train.GradientDescentOptimizer(learning_rate=1e-5)
train = optimizer.minimize(cost)
# Launch the graph in a session.
sess = tf.Session()
# Initializes global variables in the graph.
sess.run(tf.global_variables_initializer())
for step in range(2001):
cost_val, hy_val, _ = sess.run([cost, hypothesis, train],
feed_dict={x1: x1_data, x2: x2_data, x3: x3_data, Y: y_data})
if step % 10 == 0:
print(step, "Cost: ", cost_val, "\nPrediction:\n", hy_val)
'''
0 Cost: 19614.8
Prediction:
[ 21.69748688 39.10213089 31.82624626 35.14236832 32.55316544]
10 Cost: 14.0682
Prediction:
[ 145.56100464 187.94958496 178.50236511 194.86721802 146.08096313]
...
1990 Cost: 4.9197
Prediction:
[ 148.15084839 186.88632202 179.6293335 195.81796265 144.46044922]
2000 Cost: 4.89449
Prediction:
[ 148.15931702 186.8805542 179.63194275 195.81971741 144.45298767]
'''
[ 매트릭스 적용 ]
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
# Lab 4 Multi-variable linear regression
import tensorflow as tf
tf.set_random_seed(777) # for reproducibility
매트릭로 DATA 적용
x_data = [[73., 80., 75.],
[93., 88., 93.],
[89., 91., 90.],
[96., 98., 100.],
[73., 66., 70.]]
y_data = [[152.],
[185.],
[180.],
[196.],
[142.]]
# placeholders for a tensor that will be always fed.
X = tf.placeholder(tf.float32, shape=[None, 3])
Y = tf.placeholder(tf.float32, shape=[None, 1])
W = tf.Variable(tf.random_normal([3, 1]), name='weight')
b = tf.Variable(tf.random_normal([1]), name='bias')
매트릭스 곱샘 코딩
# Hypothesis
hypothesis = tf.matmul(X, W) + b
# Simplified cost/loss function
cost = tf.reduce_mean(tf.square(hypothesis - Y))
# Minimize
optimizer = tf.train.GradientDescentOptimizer(learning_rate=1e-5)
train = optimizer.minimize(cost)
# Launch the graph in a session.
sess = tf.Session()
# Initializes global variables in the graph.
sess.run(tf.global_variables_initializer())
for step in range(2001):
cost_val, hy_val, _ = sess.run(
[cost, hypothesis, train], feed_dict={X: x_data, Y: y_data})
if step % 10 == 0:
print(step, "Cost: ", cost_val, "\nPrediction:\n", hy_val)
'''
0 Cost: 7105.46
Prediction:
[[ 80.82241058]
[ 92.26364136]
[ 93.70250702]
[ 98.09217834]
[ 72.51759338]]
10 Cost: 5.89726
Prediction:
[[ 155.35159302]
[ 181.85691833]
[ 181.97254944]
[ 194.21760559]
[ 140.85707092]]
...
1990 Cost: 3.18588
Prediction:
[[ 154.36352539]
[ 182.94833374]
[ 181.85189819]
[ 194.35585022]
[ 142.03240967]]
2000 Cost: 3.1781
Prediction:
[[ 154.35881042]
[ 182.95147705]
[ 181.85035706]
[ 194.35533142]
[ 142.036026 ]]
'''
[ TENSOR FLOW로 파일에서 DATA 읽어오기 ]
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
# Lab 4 Multi-variable linear regression
import tensorflow as tf
import numpy as np
tf.set_random_seed(777) # for reproducibility
DATA는 CSV 화일에 저장됨, CSV 화일 위치는 PY 화일위치에 있음
xy = np.loadtxt('data-01-test-score.csv', delimiter=',', dtype=np.float32)
슬라이싱방법 숙지필요함
x_data = xy[:, 0:-1]
y_data = xy[:, [-1]]
# Make sure the shape and data are OK
print(x_data.shape, x_data, len(x_data))
print(y_data.shape, y_data)
# placeholders for a tensor that will be always fed.
X = tf.placeholder(tf.float32, shape=[None, 3])
Y = tf.placeholder(tf.float32, shape=[None, 1])
W = tf.Variable(tf.random_normal([3, 1]), name='weight')
b = tf.Variable(tf.random_normal([1]), name='bias')
# Hypothesis
hypothesis = tf.matmul(X, W) + b
# Simplified cost/loss function
cost = tf.reduce_mean(tf.square(hypothesis - Y))
# Minimize
optimizer = tf.train.GradientDescentOptimizer(learning_rate=1e-5)
train = optimizer.minimize(cost)
# Launch the graph in a session.
sess = tf.Session()
# Initializes global variables in the graph.
sess.run(tf.global_variables_initializer())
for step in range(2001):
cost_val, hy_val, _ = sess.run(
[cost, hypothesis, train], feed_dict={X: x_data, Y: y_data})
if step % 10 == 0:
print(step, "Cost: ", cost_val, "\nPrediction:\n", hy_val)
나의 점수 예측
# Ask my score
print("Your score will be ", sess.run(
hypothesis, feed_dict={X: [[100, 70, 101]]}))
print("Other scores will be ", sess.run(hypothesis,
feed_dict={X: [[60, 70, 110], [90, 100, 80]]}))
'''
타인의 점수 예측
Your score will be [[ 181.73277283]]
Other scores will be [[ 145.86265564]
[ 187.23129272]]
'''
[ DATA QUE로 읽어들임 시간차로 처리해서 시스템 부하를 줄어줌 ]
import tensorflow as tf
tf.set_random_seed(777) # for reproducibility
화일 하나를 위한 QUE를 만듦
filename_queue = tf.train.string_input_producer(
['data-01-test-score.csv'], shuffle=False, name='filename_queue')
화일 리더 정의
reader = tf.TextLineReader()
key, value = reader.read(filename_queue)
# Default values, in case of empty columns. Also specifies the type of the
# decoded result.
CSV로 디코딩
record_defaults = [[0.], [0.], [0.], [0.]]
xy = tf.decode_csv(value, record_defaults=record_defaults)
불러올때마다 10 개씩 불러옴 (BATCH_SIZE =10 )
# collect batches of csv in
train_x_batch, train_y_batch = \
tf.train.batch([xy[0:-1], xy[-1:]], batch_size=10)
# placeholders for a tensor that will be always fed.
X 3개 , Y 1개 데이터 숫자를 맞추어줌
X = tf.placeholder(tf.float32, shape=[None, 3])
Y = tf.placeholder(tf.float32, shape=[None, 1])
W = tf.Variable(tf.random_normal([3, 1]), name='weight')
b = tf.Variable(tf.random_normal([1]), name='bias')
# Hypothesis
hypothesis = tf.matmul(X, W) + b
# Simplified cost/loss function
cost = tf.reduce_mean(tf.square(hypothesis - Y))
# Minimize
optimizer = tf.train.GradientDescentOptimizer(learning_rate=1e-5)
train = optimizer.minimize(cost)
# Launch the graph in a session.
sess = tf.Session()
# Initializes global variables in the graph.
sess.run(tf.global_variables_initializer())
# Start populating the filename queue.
coord = tf.train.Coordinator()
threads = tf.train.start_queue_runners(sess=sess, coord=coord)
for step in range(2001):
x_batch , y_batch 값을 train feed_dict로 넘겨줌
x_batch, y_batch = sess.run([train_x_batch, train_y_batch])
cost_val, hy_val, _ = sess.run(
[cost, hypothesis, train], feed_dict={X: x_batch, Y: y_batch})
if step % 10 == 0:
print(step, "Cost: ", cost_val, "\nPrediction:\n", hy_val)
coord.request_stop()
coord.join(threads)
# Ask my score
print("Your score will be ",
sess.run(hypothesis, feed_dict={X: [[100, 70, 101]]}))
print("Other scores will be ",
sess.run(hypothesis, feed_dict={X: [[60, 70, 110], [90, 100, 80]]}))
'''
Your score will be [[ 177.78144836]]
Other scores will be [[ 141.10997009]
[ 191.17378235]]
'''
[참고자료 ]
https://www.inflearn.com/course/기본적인-머신러닝-딥러닝-강좌/
http://agiantmind.tistory.com/176
https://www.tensorflow.org/install/
https://github.com/hunkim/deeplearningzerotoall
http://www.derivative-calculator.net/
http://terms.naver.com/entry.nhn?docId=3350391&cid=58247&categoryId=58247 ==> 미분계산/공식
http://matplotlib.org/users/installing.html ==>matplotlib 설치
'프로젝트 > 인공지능' 카테고리의 다른 글
| [인공지능 #6 ] multinomial 적용 (0) | 2017.07.30 |
|---|---|
| [인공지능 #5 ] Logistics classification 가설함수 (0) | 2017.07.30 |
| [인공지능 #3 ] Cost 최소화 알고리즘 구현 (0) | 2017.07.30 |
| [인공지능 #2 ] Tensorflow Linear regression 구현 (0) | 2017.07.30 |
| [인공지능 #1 ] Tensorflow를 실행하기 위한 사전준비 (0) | 2017.07.29 |
[인공지능 #3 ] Cost 최소화 알고리즘 구현
인공지능 구현에 대한 글입니다.
글의 순서는 아래와 같습니다.
================================================
1. Cost 최소화 알고리즘 구현 소스코드 분석
2. Cost Minimize 코딩 작성 ( cost 를 미분해서 W변화량 적용)
3. Optimizer 적용 테스트
- 간단한 미분은 상기과 같이 직접코딩이 가능하겠으나, 복잡해지면 통상적으로 optimizer로 코딩함
4. Optimizer 적용 과 수작업(미분계산) 수치 비교
- 경사도 수치를 중간에 변형할수 있도록 코딩함 (gvs) ==> 거의 같음
5. Next step
==> multivariable linear regresstion : 하나이상의 변수적용
6. 참고자료
=================================================
[ Cost 최소화 알고리즘 구현 소스코드 분석 ]
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
# Lab 3 Minimizing Cost
import tensorflow as tf
import matplotlib.pyplot as plt
==> matplotlib를 사용키 위해서는 별도로 설치를 해야함
==> http://matplotlib.org/users/installing.html
==> 그래프를 그려주는 기능임.
tf.set_random_seed(777) # for reproducibility
X = [1, 2, 3]
Y = [1, 2, 3]
W = tf.placeholder(tf.float32)
# Our hypothesis for linear model X * W
hypothesis = X * W
# cost/loss function
cost = tf.reduce_mean(tf.square(hypothesis - Y))
# Launch the graph in a session.
sess = tf.Session()
# Variables for plotting cost function
W_history = []
cost_history = []
==> 그래프를 그리기 위해 배열할당
for i in range(-30, 50):
curr_W = i * 0.1
curr_cost = sess.run(cost, feed_dict={W: curr_W})
W_history.append(curr_W)
cost_history.append(curr_cost)
# Show the cost function
plt.plot(W_history, cost_history)
plt.show()
[ Cost Minimize 코딩 작성 ( cost 를 미분해서 W변화량 적용) ]
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
# Lab 3 Minimizing Cost
import tensorflow as tf
tf.set_random_seed(777) # for reproducibility
x_data = [1, 2, 3]
y_data = [1, 2, 3]
# Try to find values for W and b to compute y_data = W * x_data + b
# We know that W should be 1 and b should be 0
# But let's use TensorFlow to figure it out
W = tf.Variable(tf.random_normal([1]), name='weight')
X = tf.placeholder(tf.float32)
Y = tf.placeholder(tf.float32)
# Our hypothesis for linear model X * W
hypothesis = X * W
# cost/loss function
cost = tf.reduce_mean(tf.square(hypothesis - Y))
W값의 descent 간격을 구하는 과정을 미분방정식으로 직접(수작업) 코딩하는 내용입니다( 아래그림은 미분적용 과정입니다)
# Minimize: Gradient Descent using derivative: W -= learning_rate * derivative
learning_rate = 0.1
gradient = tf.reduce_mean((W * X - Y) * X)
descent = W - learning_rate * gradient
update = W.assign(descent)
간단한 미분은 상기과 같이 직접코딩이 가능하겠으나, 복잡해지면 통상적으로 아래와 같이 코딩을 하는것이 일반적임(텐서플로우가 자동계산을 해줌)
# Launch the graph in a session.
sess = tf.Session()
# Initializes global variables in the graph.
sess.run(tf.global_variables_initializer())
for step in range(21):
sess.run(update, feed_dict={X: x_data, Y: y_data})
print(step, sess.run(cost, feed_dict={X: x_data, Y: y_data}), sess.run(W))
'''
0 1.93919 [ 1.64462376]
1 0.551591 [ 1.34379935]
2 0.156897 [ 1.18335962]
3 0.0446285 [ 1.09779179]
4 0.0126943 [ 1.05215561]
5 0.00361082 [ 1.0278163]
6 0.00102708 [ 1.01483536]
7 0.000292144 [ 1.00791216]
8 8.30968e-05 [ 1.00421977]
9 2.36361e-05 [ 1.00225055]
10 6.72385e-06 [ 1.00120032]
11 1.91239e-06 [ 1.00064015]
12 5.43968e-07 [ 1.00034142]
13 1.54591e-07 [ 1.00018203]
14 4.39416e-08 [ 1.00009704]
15 1.24913e-08 [ 1.00005174]
16 3.5322e-09 [ 1.00002754]
17 9.99824e-10 [ 1.00001466]
18 2.88878e-10 [ 1.00000787]
19 8.02487e-11 [ 1.00000417]
20 2.34053e-11 [ 1.00000226]
'''
[ Optimizer 적용 테스트 ]
import os os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3' # Lab 3 Minimizing Cost import tensorflow as tf tf.set_random_seed(777) # for reproducibility # tf Graph Input X = [1, 2, 3] Y = [1, 2, 3] # Set wrong model weights W = tf.Variable(5.0) ==> w 값을 터무니없는 값을 주고, 적정한 값을 찾아가는지 테스트함 # Linear model hypothesis = X * W # cost/loss function cost = tf.reduce_mean(tf.square(hypothesis - Y)) # Minimize: Gradient Descent Magic optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.1) train = optimizer.minimize(cost) # Launch the graph in a session. sess = tf.Session() # Initializes global variables in the graph. sess.run(tf.global_variables_initializer()) for step in range(100): print(step, sess.run(W)) sess.run(train) ''' 0 5.0 1 1.26667 2 1.01778 3 1.00119 4 1.00008 ... 96 1.0 97 1.0 98 1.0 99 1.0 '''
[ Optimizer 적용 과 수작업(미분계산) 수치 비교]
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
# Lab 3 Minimizing Cost
# This is optional
import tensorflow as tf
tf.set_random_seed(777) # for reproducibility
# tf Graph Input
X = [1, 2, 3]
Y = [1, 2, 3]
# Set wrong model weights
W = tf.Variable(5.)
# Linear model
hypothesis = X * W
# Manual gradient
gradient = tf.reduce_mean((W * X - Y) * X) * 2
# cost/loss function
cost = tf.reduce_mean(tf.square(hypothesis - Y))
# Minimize: Gradient Descent Magic
optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.01)
train = optimizer.minimize(cost)
# Get gradients
gvs = optimizer.compute_gradients(cost, [W])
# Optional: modify gradient if necessary
# gvs = [(tf.clip_by_value(grad, -1., 1.), var) for grad, var in gvs]
==> gvc 수치를 필요에 따라서 변형적용이 가능함
# Apply gradients
apply_gradients = optimizer.apply_gradients(gvs)
# Launch the graph in a session.
sess = tf.Session()
# Initializes global variables in the graph.
sess.run(tf.global_variables_initializer())
for step in range(100):
print(step, sess.run([gradient, W, gvs]))
sess.run(apply_gradients)
# Same as sess.run(train)
'''
# Apply gradients
0 [37.333332, 5.0, [(37.333336, 5.0)]]
1 [33.848888, 4.6266665, [(33.848888, 4.6266665)]]
2 [30.689657, 4.2881775, [(30.689657, 4.2881775)]]
3 [27.825287, 3.9812808, [(27.825287, 3.9812808)]]
4 [25.228262, 3.703028, [(25.228264, 3.703028)]]
...
96 [0.0030694802, 1.0003289, [(0.0030694804, 1.0003289)]]
97 [0.0027837753, 1.0002983, [(0.0027837753, 1.0002983)]]
98 [0.0025234222, 1.0002704, [(0.0025234222, 1.0002704)]]
99 [0.0022875469, 1.0002451, [(0.0022875469, 1.0002451)]]
'''
[ 참고자료 ]
https://www.inflearn.com/course/기본적인-머신러닝-딥러닝-강좌/
http://agiantmind.tistory.com/176
https://www.tensorflow.org/install/
https://github.com/hunkim/deeplearningzerotoall
http://www.derivative-calculator.net/
http://terms.naver.com/entry.nhn?docId=3350391&cid=58247&categoryId=58247 ==> 미분계산/공식
http://matplotlib.org/users/installing.html ==>matplotlib 설치
'프로젝트 > 인공지능' 카테고리의 다른 글
| [인공지능 #6 ] multinomial 적용 (0) | 2017.07.30 |
|---|---|
| [인공지능 #5 ] Logistics classification 가설함수 (0) | 2017.07.30 |
| [인공지능 #4 ] 여러개의 DATA를 (X 변수) 구현 (0) | 2017.07.30 |
| [인공지능 #2 ] Tensorflow Linear regression 구현 (0) | 2017.07.30 |
| [인공지능 #1 ] Tensorflow를 실행하기 위한 사전준비 (0) | 2017.07.29 |
