[인공지능 #7] Rate overfiting , training/test data , nomalization
인공지능 구현에 대한 글입니다.
글의 순서는 아래와 같습니다.
================================================
1. rate overfitting , regularization tips
2. train/test data sheet , learning rate , normalization(new)
- 학습과 테스트 data를 구분하는것이 합리적임
- mnist 소개
3. 참고자료
=================================================
[rate overfitting , regularization tips]
1. COST 값이 줄지않고, 늘어난다면 Learning rate을 더 작게 변경해주어야함
반대로 너무 작게 줄거나, 도중에 멈춘다면 learing rate을 좀더 크게 해주어야함
보통 0.01을 기준으로 늘리거나 줄이면서 조정해 나가면 된다.
2. x data 값이 차이가 큰 경우, cost가 잘 줄지않거나, 학습이 잘 일어나지 않음 , 이럴때는 NOMALIZATION 을 해주어야 함.
3. OVERFITTING
- 더많은 TRAINING DATA
- REDUCE FEATURES
- REGULARIZATION ==> 구별선이 구부러지지 않토록 , X값을 일반화 시킴
[train/test data sheet , learning rate , normalization(new)]
# <lab-07-1-learning_rate_and_evaluation>
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
# Lab 7 Learning rate and Evaluation
import tensorflow as tf
tf.set_random_seed(777) # for reproducibility
x_data = [[1, 2, 1],
[1, 3, 2],
[1, 3, 4],
[1, 5, 5],
[1, 7, 5],
[1, 2, 5],
[1, 6, 6],
[1, 7, 7]]
y_data = [[0, 0, 1],
[0, 0, 1],
[0, 0, 1],
[0, 1, 0],
[0, 1, 0],
[0, 1, 0],
[1, 0, 0],
[1, 0, 0]]
# Evaluation our model using this test dataset
x_test = [[2, 1, 1],
[3, 1, 2],
[3, 3, 4]]
y_test = [[0, 0, 1],
[0, 0, 1],
[0, 0, 1]]
X = tf.placeholder("float", [None, 3])
Y = tf.placeholder("float", [None, 3])
W = tf.Variable(tf.random_normal([3, 3]))
b = tf.Variable(tf.random_normal([3]))
# tf.nn.softmax computes softmax activations
# softmax = exp(logits) / reduce_sum(exp(logits), dim)
hypothesis = tf.nn.softmax(tf.matmul(X, W) + b)
# Cross entropy cost/loss
cost = tf.reduce_mean(-tf.reduce_sum(Y * tf.log(hypothesis), axis=1))
# Try to change learning_rate to small numbers
optimizer = tf.train.GradientDescentOptimizer(
learning_rate=0.1).minimize(cost)
# dhp Learning rate을 1.5로 움직이는 크기를 크게하면, H(y)값은 발산을 하게된다 : overfitting )
# Correct prediction Test model
prediction = tf.arg_max(hypothesis, 1)
is_correct = tf.equal(prediction, tf.arg_max(Y, 1))
accuracy = tf.reduce_mean(tf.cast(is_correct, tf.float32))
# Launch graph
with tf.Session() as sess:
# Initialize TensorFlow variables
sess.run(tf.global_variables_initializer())
for step in range(201):
cost_val, W_val, _ = sess.run(
[cost, W, optimizer], feed_dict={X: x_data, Y: y_data})
print(step, cost_val, W_val)
# 여기부터는 test data 로 test 진행함
# tf 입장에서는 처움보는 data (x)가 되는것이다.
# predict
print("Prediction:", sess.run(prediction, feed_dict={X: x_test}))
# Calculate the accuracy
print("Accuracy: ", sess.run(accuracy, feed_dict={X: x_test, Y: y_test}))
'''
when lr = 1.5
0 5.73203 [[-0.30548954 1.22985029 -0.66033536]
[-4.39069986 2.29670858 2.99386835]
[-3.34510708 2.09743214 -0.80419564]]
1 23.1494 [[ 0.06951046 0.29449689 -0.0999819 ]
[-1.95319986 -1.63627958 4.48935604]
[-0.90760708 -1.65020132 0.50593793]]
2 27.2798 [[ 0.44451016 0.85699677 -1.03748143]
[ 0.48429942 0.98872018 -0.57314301]
[ 1.52989244 1.16229868 -4.74406147]]
3 8.668 [[ 0.12396193 0.61504567 -0.47498202]
[ 0.22003263 -0.2470119 0.9268558 ]
[ 0.96035379 0.41933775 -3.43156195]]
4 5.77111 [[-0.9524312 1.13037777 0.08607888]
[-3.78651619 2.26245379 2.42393875]
[-3.07170963 3.14037919 -2.12054014]]
5 inf [[ nan nan nan]
[ nan nan nan]
[ nan nan nan]]
6 nan [[ nan nan nan]
[ nan nan nan]
[ nan nan nan]]
...
Prediction: [0 0 0]
Accuracy: 0.0
-------------------------------------------------
When lr = 1e-10
0 5.73203 [[ 0.80269563 0.67861295 -1.21728313]
[-0.3051686 -0.3032113 1.50825703]
[ 0.75722361 -0.7008909 -2.10820389]]
1 5.73203 [[ 0.80269563 0.67861295 -1.21728313]
[-0.3051686 -0.3032113 1.50825703]
[ 0.75722361 -0.7008909 -2.10820389]]
2 5.73203 [[ 0.80269563 0.67861295 -1.21728313]
[-0.3051686 -0.3032113 1.50825703]
[ 0.75722361 -0.7008909 -2.10820389]]
...
198 5.73203 [[ 0.80269563 0.67861295 -1.21728313]
[-0.3051686 -0.3032113 1.50825703]
[ 0.75722361 -0.7008909 -2.10820389]]
199 5.73203 [[ 0.80269563 0.67861295 -1.21728313]
[-0.3051686 -0.3032113 1.50825703]
[ 0.75722361 -0.7008909 -2.10820389]]
200 5.73203 [[ 0.80269563 0.67861295 -1.21728313]
[-0.3051686 -0.3032113 1.50825703]
[ 0.75722361 -0.7008909 -2.10820389]]
Prediction: [0 0 0]
Accuracy: 0.0
-------------------------------------------------
When lr = 0.1
0 5.73203 [[ 0.72881663 0.71536207 -1.18015325]
[-0.57753736 -0.12988332 1.60729778]
[ 0.48373488 -0.51433605 -2.02127004]]
1 3.318 [[ 0.66219079 0.74796319 -1.14612854]
[-0.81948912 0.03000021 1.68936598]
[ 0.23214608 -0.33772916 -1.94628811]]
2 2.0218 [[ 0.64342022 0.74127686 -1.12067163]
[-0.81161296 -0.00900121 1.72049117]
[ 0.2086665 -0.35079569 -1.909742 ]]
...
199 0.672261 [[-1.15377033 0.28146935 1.13632679]
[ 0.37484586 0.18958236 0.33544877]
[-0.35609841 -0.43973011 -1.25604188]]
200 0.670909 [[-1.15885413 0.28058422 1.14229572]
[ 0.37609792 0.19073224 0.33304682]
[-0.35536593 -0.44033223 -1.2561723 ]]
Prediction: [2 2 2]
Accuracy: 1.0
'''
# < lab-07-2-linear_regression_without_min_max >
# dhp data 편차가 터무니없이 크게되면, 학습이 잘 되질 않됨
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
import tensorflow as tf
import numpy as np
tf.set_random_seed(777) # for reproducibility
xy = np.array([[828.659973, 833.450012, 908100, 828.349976, 831.659973],
[823.02002, 828.070007, 1828100, 821.655029, 828.070007],
[819.929993, 824.400024, 1438100, 818.97998, 824.159973],
[816, 820.958984, 1008100, 815.48999, 819.23999],
[819.359985, 823, 1188100, 818.469971, 818.97998],
[819, 823, 1198100, 816, 820.450012],
[811.700012, 815.25, 1098100, 809.780029, 813.669983],
[809.51001, 816.659973, 1398100, 804.539978, 809.559998]])
x_data = xy[:, 0:-1]
y_data = xy[:, [-1]]
# placeholders for a tensor that will be always fed.
X = tf.placeholder(tf.float32, shape=[None, 4])
Y = tf.placeholder(tf.float32, shape=[None, 1])
W = tf.Variable(tf.random_normal([4, 1]), name='weight')
b = tf.Variable(tf.random_normal([1]), name='bias')
# Hypothesis
hypothesis = tf.matmul(X, W) + b
# Simplified cost/loss function
cost = tf.reduce_mean(tf.square(hypothesis - Y))
# Minimize
optimizer = tf.train.GradientDescentOptimizer(learning_rate=1e-5)
train = optimizer.minimize(cost)
# Launch the graph in a session.
sess = tf.Session()
# Initializes global variables in the graph.
sess.run(tf.global_variables_initializer())
for step in range(101):
cost_val, hy_val, _ = sess.run(
[cost, hypothesis, train], feed_dict={X: x_data, Y: y_data})
print(step, "Cost: ", cost_val, "\nPrediction:\n", hy_val)
'''
0 Cost: 2.45533e+12
Prediction:
[[-1104436.375]
[-2224342.75 ]
[-1749606.75 ]
[-1226179.375]
[-1445287.125]
[-1457459.5 ]
[-1335740.5 ]
[-1700924.625]]
1 Cost: 2.69762e+27
Prediction:
[[ 3.66371490e+13]
[ 7.37543360e+13]
[ 5.80198785e+13]
[ 4.06716290e+13]
[ 4.79336847e+13]
[ 4.83371348e+13]
[ 4.43026590e+13]
[ 5.64060907e+13]]
2 Cost: inf
Prediction:
[[ -1.21438790e+21]
[ -2.44468702e+21]
[ -1.92314724e+21]
[ -1.34811610e+21]
[ -1.58882674e+21]
[ -1.60219962e+21]
[ -1.46847142e+21]
[ -1.86965602e+21]]
3 Cost: inf
Prediction:
[[ 4.02525216e+28]
[ 8.10324465e+28]
[ 6.37453079e+28]
[ 4.46851237e+28]
[ 5.26638074e+28]
[ 5.31070676e+28]
[ 4.86744608e+28]
[ 6.19722623e+28]]
4 Cost: inf
Prediction:
[[ -1.33422428e+36]
[ -2.68593010e+36]
[ -2.11292430e+36]
[ -1.48114879e+36]
[ -1.74561303e+36]
[ -1.76030542e+36]
[ -1.61338091e+36]
[ -2.05415459e+36]]
5 Cost: inf
Prediction:
[[ inf]
[ inf]
[ inf]
[ inf]
[ inf]
[ inf]
[ inf]
[ inf]]
6 Cost: nan
Prediction:
[[ nan]
[ nan]
[ nan]
[ nan]
[ nan]
[ nan]
[ nan]
[ nan]]
'''
# <lab-07-3-linear_regression_min_max >
# dhp minmaxscala 함수를 이용해서, 정규화 (0~1사이의수)로 변환을 시킨다(nomallize)
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
import tensorflow as tf
import numpy as np
tf.set_random_seed(777) # for reproducibility
def MinMaxScaler(data):
numerator = data - np.min(data, 0)
denominator = np.max(data, 0) - np.min(data, 0)
# noise term prevents the zero division
return numerator / (denominator + 1e-7)
xy = np.array([[828.659973, 833.450012, 908100, 828.349976, 831.659973],
[823.02002, 828.070007, 1828100, 821.655029, 828.070007],
[819.929993, 824.400024, 1438100, 818.97998, 824.159973],
[816, 820.958984, 1008100, 815.48999, 819.23999],
[819.359985, 823, 1188100, 818.469971, 818.97998],
[819, 823, 1198100, 816, 820.450012],
[811.700012, 815.25, 1098100, 809.780029, 813.669983],
[809.51001, 816.659973, 1398100, 804.539978, 809.559998]])
# very important. It does not work without it.
xy = MinMaxScaler(xy)
print(xy)
x_data = xy[:, 0:-1]
y_data = xy[:, [-1]]
# placeholders for a tensor that will be always fed.
X = tf.placeholder(tf.float32, shape=[None, 4])
Y = tf.placeholder(tf.float32, shape=[None, 1])
W = tf.Variable(tf.random_normal([4, 1]), name='weight')
b = tf.Variable(tf.random_normal([1]), name='bias')
# Hypothesis
hypothesis = tf.matmul(X, W) + b
# Simplified cost/loss function
cost = tf.reduce_mean(tf.square(hypothesis - Y))
# Minimize
optimizer = tf.train.GradientDescentOptimizer(learning_rate=1e-5)
train = optimizer.minimize(cost)
# Launch the graph in a session.
sess = tf.Session()
# Initializes global variables in the graph.
sess.run(tf.global_variables_initializer())
for step in range(101):
cost_val, hy_val, _ = sess.run(
[cost, hypothesis, train], feed_dict={X: x_data, Y: y_data})
print(step, "Cost: ", cost_val, "\nPrediction:\n", hy_val)
'''
100 Cost: 0.152254
Prediction:
[[ 1.63450289]
[ 0.06628087]
[ 0.35014752]
[ 0.67070574]
# <lab-07-4-mnist_introduction>
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
# Lab 7 Learning rate and Evaluation
import tensorflow as tf
import random
# import matplotlib.pyplot as plt
tf.set_random_seed(777) # for reproducibility
from tensorflow.examples.tutorials.mnist import input_data
# Check out https://www.tensorflow.org/get_started/mnist/beginners for
# more information about the mnist dataset
mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)
nb_classes = 10
# MNIST data image of shape 28 * 28 = 784
X = tf.placeholder(tf.float32, [None, 784])
# 0 - 9 digits recognition = 10 classes
Y = tf.placeholder(tf.float32, [None, nb_classes])
W = tf.Variable(tf.random_normal([784, nb_classes]))
b = tf.Variable(tf.random_normal([nb_classes]))
# Hypothesis (using softmax)
hypothesis = tf.nn.softmax(tf.matmul(X, W) + b)
cost = tf.reduce_mean(-tf.reduce_sum(Y * tf.log(hypothesis), axis=1))
optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.1).minimize(cost)
# Test model
is_correct = tf.equal(tf.arg_max(hypothesis, 1), tf.arg_max(Y, 1))
# Calculate accuracy
accuracy = tf.reduce_mean(tf.cast(is_correct, tf.float32))
# parameters
training_epochs = 15
batch_size = 100
with tf.Session() as sess:
# Initialize TensorFlow variables
sess.run(tf.global_variables_initializer())
# Training cycle
for epoch in range(training_epochs):
avg_cost = 0
total_batch = int(mnist.train.num_examples / batch_size)
for i in range(total_batch):
batch_xs, batch_ys = mnist.train.next_batch(batch_size)
c, _ = sess.run([cost, optimizer], feed_dict={
X: batch_xs, Y: batch_ys})
avg_cost += c / total_batch
print('Epoch:', '%04d' % (epoch + 1),
'cost =', '{:.9f}'.format(avg_cost))
print("Learning finished")
# Test the model using test sets
print("Accuracy: ", accuracy.eval(session=sess, feed_dict={
X: mnist.test.images, Y: mnist.test.labels}))
# Get one and predict
r = random.randint(0, mnist.test.num_examples - 1)
print("Label: ", sess.run(tf.argmax(mnist.test.labels[r:r + 1], 1)))
print("Prediction: ", sess.run(
tf.argmax(hypothesis, 1), feed_dict={X: mnist.test.images[r:r + 1]}))
# 그래프 인쇄가 않됨, 향후 원인 파악필요함, 일단은 #로 주석처리 함
# don't know why this makes Travis Build error.
# plt.imshow(
# mnist.test.images[r:r + 1].reshape(28, 28),
# cmap='Greys',
# interpolation='nearest')
# plt.show()
'''
Epoch: 0001 cost = 2.868104637
Epoch: 0002 cost = 1.134684615
Epoch: 0003 cost = 0.908220728
Epoch: 0004 cost = 0.794199896
Epoch: 0005 cost = 0.721815854
Epoch: 0006 cost = 0.670184430
Epoch: 0007 cost = 0.630576546
Epoch: 0008 cost = 0.598888191
Epoch: 0009 cost = 0.573027079
Epoch: 0010 cost = 0.550497213
[참고자료 ]
'프로젝트 > 인공지능' 카테고리의 다른 글
[인공지능 #9]mnist_softmax /mnist_nn/mnist_nn_xavier/mnist_nn_deep / mnist_nn_dropout (0) | 2017.08.06 |
---|---|
[인공지능 #8]xor / xor-nn / xor-nn-wide-deep (0) | 2017.08.06 |
[python] python 문법정리 (0) | 2017.08.05 |
[인공지능 #6 ] multinomial 적용 (0) | 2017.07.30 |
[인공지능 #5 ] Logistics classification 가설함수 (0) | 2017.07.30 |
[python] python 문법정리
python 문법에 관한 글 입니다.
인공지능 tensor flow 코딩을 위해 python 문법에 대해 정리를 하고자 합니다.
글의 순서는 아래와 같습니다
==========================================
1. cheatsheet
- python문법의 컨닝 페이퍼입니다.
2. 주제별 코딩예제 모음
3. 참고한 자료
==========================================
[cheatsheet]
[주제별 코딩예제 모음]
print('---수와 계산------------------------------------------------------------------------')
print(10 + 5)
print(10 - 5)
print(10 * 5)
print(10 / 5)
import math
print(math.ceil(2.2))
print(math.floor(2.7))
print(math.pow(2,10))
print(math.pi)
print('---문자와 데이터 타입----------------------------------------------------------------')
print('Hello')
print("Hello")
print("Hello 'world'")
print('Hello "world"')
print('Hello '+'world')
print('Hello '*3)
print('Hello'[0])
print('Hello'[1])
print('Hello'[2])
print('hello world'.capitalize())
print('hello world'.upper())
print('hello world'.__len__())
print(len('hello world'))
print('Hello world'.replace('world', 'programming'))
print("egoing's \"tutorial\"")
print("\\")
print("Hello\nworld")
print("Hello\t\tworld")
print("\a")
print('Hello\nworld')
print(10+5)
print("10"+"5")
print('---변수---------------------------------------------------------------------------')
x = 10
y = 5
print(x + y)
title = "python & ruby"
print("Title is " + title)
print('---문자열에서 변수의 사용---------------------------------------------------------------------------')
name = "이상효"
print("안녕하세요. "+name+"님")
print(name+"님을 위한 강의를 준비했습니다.")
print(name+"님 꼭 참석 부탁드립니다.")
print('---수계산 에서 변수의 사용---------------------------------------------------------------------------')
donation = 200
student = 10
sponsor = 100
print((donation*student)/sponsor)
print('---비교와 불리언---------------------------------------------------------------------------')
a=1
b=1
print(a==b)
print(1==2)
print(1>2)
print(1<2)
print(True)
print(False)
print('---조건문---------------------------------------------------------------------------')
if True:
print("code1")
print("code2")
print("code3")
print('---조건문의 활용---------------------------------------------------------------------------')
input = 11
real = 11
if real == input:
print("Hello!")
input = 11
real = 21
if real == input:
print("Hello!")
else:
print("Who are you?")
input = 33
real_egoing = 11
real_k8805 = "ab"
if real_egoing == input:
print("Hello!, egoing")
elif real_k8805 == input:
print("Hello!, k8805")
else:
print("Who are you?")
print('---입력과 출력---------------------------------------------------------------------------')
in_str = input("입력 해주세요.\n")
print(in_str.upper()+" World!")
print('---로그인 애플리케이션에 입력기능 추가하기-------------------------------------------------------------------------')
in_str = input("1 아이디를 입력해주세요.\n")
real_egoing = "11"
real_k8805 = "ab"
if real_egoing == in_str:
print("Hello!, egoing")
elif real_k8805 == in_str:
print("Hello!, k8805")
else:
print("Who are you?")
print('---논리 연산---------------------------------------------------------------------------')
in_str = input("2 아이디를 입력해주세요.\n")
real_egoing = "egoing"
real_k8805 = "k8805"
if real_egoing == in_str or real_k8805 == in_str:
print("Hello!")
else:
print("Who are you?")
input_id = input("아이디를 입력해주세요.\n")
input_pwd = input("비밀번호를 입력해주세요.\n")
real_id = "egoing"
real_pwd = "11"
if real_id == input_id:
if real_pwd == input_pwd:
print("Hello!")
else:
print("잘못된 비밀번호입니다")
else:
print("잘못된 아이디입니다")
input_id = input("아이디를 입력해주세요.\n")
input_pwd = input("비밀번호를 입력해주세요.\n")
real_id = "egoing"
real_pwd = "11"
if real_id == input_id and real_pwd == input_pwd:
print("Hello!")
else:
print("로그인에 실패했습니다")
print('---주석---------------------------------------------------------------------------')
'''
조건문 예제
egoing
2015
'''
# user input password
input = 33
real_egoing = 11
#real_k8805 = "ab"
if real_egoing == input:
print("Hello!, egoing")
#elif real_k8805 == input:
# print("Hello!, k8805")
else:
print("Who are you?")
print('---컨테이너---------------------------------------------------------------------------')
print(type('egoing')) # <class 'str'>
name = 'egoing'
print(name) # egoing
print(type(['egoing', 'leezche', 'graphittie'])) # <class 'list'>
names = ['egoing', 'leezche', 'graphittie']
print(names)
print(names[2]) # graphittie
egoing = ['programmer', 'seoul', 25, False]
egoing[1] = 'busan'
print(egoing) # ['programmer', 'busan', 25, False]
print('---사용설명서---------------------------------------------------------------------------')
al = ['A', 'B', 'C', 'D']
print(len(al)) # 4
al.append('E')
print(al) # ['A', 'B', 'C', 'D', 'E']
del (al[0])
print(al) # ['B', 'C', 'D', 'E']
print('---반복문---------------------------------------------------------------------------')
while False:
print('Hello world')
print('After while')
i = 0
while i < 3:
print('Hello world')
i = i + 1
i = 0
while i < 10:
print('print("Hello world ' + str(i * 9) + '")')
i = i + 1
i = 0
while i < 10:
if i == 4:
print(i)
i = i + 1
print('---컨테이너와 반복문-------------------------------------------------------------------------')
members = ['egoing', 'leezche', 'graphittie']
i = 0
while i < len(members):
print(members[i])
i = i + 1
members = ['egoing', 'leezche', 'graphittie']
for member in members:
print(member)
for item in range(5, 11):
print(item)
input_id = input("아이디를 입력해주세요.\n")
members = ['egoing', 'k8805', 'leezche']
for member in members:
if member == input_id:
print('Hello!, ' + member)
import sys
sys.exit()
print('Who are you?')
print('---함수---------------------------------------------------------------------------')
def a3():
print('aaa')
a3()
print('------------------------------------------------------------------------------------------')
def a3():
return 'aaa'
print(a3())
print('------------------------------------------------------------------------------------------')
def a(num):
return 'a' * num
print(a(3))
print('------------------------------------------------------------------------------------------')
input_id = input("아이디를 입력해주세요.\n")
def login(_id):
members = ['egoing', 'k8805', 'leezche']
for member in members:
if member == _id:
return True
return False
if login(input_id):
print('Hello, ' + input_id)
else:
print('Who are you?')
print('---모듈-------------------------------------------------------------------------------')
import math
print(math.ceil(2.9))
print(math.floor(2.9))
print(math.sqrt(16))
print('----모듈이 없을때-----------------------------------------------------------------------------------')
def a():
return 'B'
from egoing import a as z
import k8805 as k
print(z())
print(k.a())
print('---객체제작_객체사용---------------------------------------------------------------------------')
class Cal(object):
def __init__(self, v1, v2):
self.v1 = v1
self.v2 = v2
def add(self):
return self.v1 + self.v2
def subtract(self):
return self.v1 - self.v2
c1 = Cal(10, 10)
print(c1.add())
print(c1.subtract())
c2 = Cal(30, 20)
print(c2.add())
print(c2.subtract())
print('------객체제작_클래스-----------------------------------------------------------------')
class Cal(object):
c1 = Cal(10, 10)
print(c1.add())
print(c1.subtract())
c2 = Cal(30, 20)
print(c2.add())
print(c2.subtract())
print('------객체제작_생성자-----------------------------------------------------------------')
class Cal(object):
def __init__(self, v1, v2):
print(v1, v2)
c1 = Cal(10, 10)
# print(c1.add())
# print(c1.subtract())
c2 = Cal(30, 20)
# print(c2.add())
# print(c2.subtract())
print('------객체제작_인스턴스 변수와 메소드-----------------------------------------------------------------')
class Cal(object):
def __init__(self, v1, v2):
self.v1 = v1
self.v2 = v2
def add(self):
return self.v1 + self.v2
def subtract(self):
return self.v1 - self.v2
c1 = Cal(10, 10)
print(c1.add())
print(c1.subtract())
c2 = Cal(30, 20)
print(c2.add())
print(c2.subtract())
print('------객체와변수_인스턴스 변수의 특성-----------------------------------------------------------------')
class C(object):
def __init__(self, v):
self.value = v
def show(self):
print(self.value)
c1 = C(10)
print(c1.value)
c1.value = 20
print(c1.value)
c1.show()
print('------객체와변수_set/get 메소드--------------------------------------------------------------')
class C(object):
def __init__(self, v):
self.value = v
def show(self):
print(self.value)
def getValue(self):
return self.value
def setValue(self, v):
self.value = v
c1 = C(10)
print(c1.getValue())
c1.setValue(20)
print(c1.getValue())
print('------객체와변수_set/get 메소드를 사용해야하는 이유--------------------------------------------------')
class Cal(object):
def __init__(self, v1, v2):
if isinstance(v1, int):
self.v1 = v1
if isinstance(v2, int):
self.v2 = v2
def add(self):
return self.v1 + self.v2
def subtract(self):
return self.v1 - self.v2
def setV1(self, v):
if isinstance(v, int): # v 가 int class의 인스턴스인지 확인
self.v1 = v
def getV1(self):
return self.v1
c1 = Cal(10, 10)
print(c1.add())
print(c1.subtract())
c1.setV1('one')
c1.v2 = 30
print(c1.add())
print(c1.subtract())
print('---------객체와변수_속성 : 실행않됨---------------------------------------------------------------------')
"""
class C(object):
def __init__(self, v):
self.__value = v
def show(self):
print(self.__value)
c1 = C(10)
print(c1.__value)
c1.show()
"""
print('---상속_상속의 문법---------------------------------------------------------------------------')
class Class1(object):
def method1(self): return 'm1'
c1 = Class1()
print(c1, c1.method1())
class Class3(Class1):
def method2(self): return 'm2'
c3 = Class3()
print(c3, c3.method1())
print(c3, c3.method2())
class Class2(object):
def method1(self): return 'm1'
def method2(self): return 'm2'
c2 = Class2()
print(c2, c2.method1())
print(c2, c2.method2())
print('-----상속_상속의 응용-------------------------------------------------------------------')
"""
클래스의 재활용/함수추가 : 타인의 또는 자신의 클래스를 상속하고, 새로운 기능추가, 상속은 변수까지도 이어받음
"""
class Cal(object):
def __init__(self, v1, v2):
if isinstance(v1, int):
self.v1 = v1
if isinstance(v2, int):
self.v2 = v2
def add(self):
return self.v1 + self.v2
def subtract(self):
return self.v1 - self.v2
def setV1(self, v):
if isinstance(v, int):
self.v1 = v
def getV1(self):
return self.v1
class CalMultiply(Cal):
def multiply(self):
return self.v1 * self.v2
class CalDivide(CalMultiply):
def divide(self):
return self.v1 / self.v2
c1 = CalMultiply(10, 10)
print(c1.add())
print(c1.multiply())
c2 = CalDivide(20, 10)
print(c2, c2.add())
print(c2, c2.multiply())
print(c2, c2.divide())
print('-------------clss멤버_클래스 매소드--------------------------------------------------------------')
class Cs:
@staticmethod
def static_method():
print("Static method")
@classmethod
def class_method(cls):
print("Class method")
def instance_method(self):
print("Instance method")
i = Cs()
Cs.static_method()
Cs.class_method()
i.instance_method()
print('-------------clss멤버_클래스 변수---------------------------------------------------------------')
class Cs:
count = 0
def __init__(self):
Cs.count = Cs.count + 1
@classmethod
def getCount(cls):
return Cs.count
i1 = Cs()
i2 = Cs()
i3 = Cs()
i4 = Cs()
print(Cs.getCount())
print('-------------clss멤버_clss member의 활용---------------------------------------------------------------')
class Cal(object):
_history = []
def __init__(self, v1, v2):
if isinstance(v1, int):
self.v1 = v1
if isinstance(v2, int):
self.v2 = v2
def add(self):
result = self.v1 + self.v2
Cal._history.append("add : %d+%d=%d" % (self.v1, self.v2, result))
return result
def subtract(self):
result = self.v1 - self.v2
Cal._history.append("subtract : %d-%d=%d" % (self.v1, self.v2, result))
return result
def setV1(self, v):
if isinstance(v, int):
self.v1 = v
def getV1(self):
return self.v1
@classmethod
def history(cls):
for item in Cal._history:
print(item)
class CalMultiply(Cal):
def multiply(self):
result = self.v1 * self.v2
Cal._history.append("multiply : %d*%d=%d" % (self.v1, self.v2, result))
return result
class CalDivide(CalMultiply):
def divide(self):
result = self.v1 / self.v2
Cal._history.append("divide : %d/%d=%d" % (self.v1, self.v2, result))
return result
c1 = CalMultiply(10, 10)
print(c1.add())
print(c1.multiply())
c2 = CalDivide(20, 10)
print(c2, c2.add())
print(c2, c2.multiply())
print(c2, c2.divide())
Cal.history()
print('-------------override_형식---------------------------------------------------------------')
class C1:
def m(self):
return 'parent'
class C2(C1):
def m(self):
return super().m() + ' child'
pass
o = C2()
print(o.m())
print('-------------override_override 활용---------------------------------------------------------------')
class Cal(object):
_history = []
def __init__(self, v1, v2):
if isinstance(v1, int):
self.v1 = v1
if isinstance(v2, int):
self.v2 = v2
def add(self):
result = self.v1 + self.v2
Cal._history.append("add : %d+%d=%d" % (self.v1, self.v2, result))
return result
def subtract(self):
result = self.v1 - self.v2
Cal._history.append("subtract : %d-%d=%d" % (self.v1, self.v2, result))
return result
def setV1(self, v):
if isinstance(v, int):
self.v1 = v
def getV1(self):
return self.v1
@classmethod
def history(cls):
for item in Cal._history:
print(item)
def info(self):
return "Cal => v1 : %d, v2 : %d" % (self.v1, self.v2)
class CalMultiply(Cal):
def multiply(self):
result = self.v1 * self.v2
Cal._history.append("multiply : %d*%d=%d" % (self.v1, self.v2, result))
return result
def info(self):
return "CalMultiply => %s" % super().info()
class CalDivide(CalMultiply):
def divide(self):
result = self.v1 / self.v2
Cal._history.append("divide : %d/%d=%d" % (self.v1, self.v2, result))
return result
def info(self):
return "CalDivide => %s" % super().info()
c0 = Cal(30, 60)
print(c0.info())
c1 = CalMultiply(10, 10)
print(c1.info())
c2 = CalDivide(20, 10)
print(c2.info())
print('-------------객체와 모듈---------------------------------------------------------------')
import lib
obj = lib.A()
print(obj.a())
print('-------------다중상속_형식,단점---------------------------------------------------------------')
class C1():
def c1_m(self):
print("c1_m")
def m(self):
print("C1 m")
class C2():
def c2_m(self):
print("c2_m")
def m(self):
print("C2 m")
class C3(C2, C1):
def m(self):
print("C3 m")
c = C3()
c.c1_m()
c.c2_m()
c.m()
print(C3.__mro__)
print('-------------다중상속_다중상속의 활용---------------------------------------------------------------')
class CalMultiply():
def multiply(self):
return self.v1 * self.v2
class CalDivide():
def divide(self):
return self.v1 / self.v2
class Cal(CalMultiply, CalDivide): # 왼쪽부모가 우선순위가 높다
def __init__(self, v1, v2):
if isinstance(v1, int):
self.v1 = v1
if isinstance(v2, int):
self.v2 = v2
def add(self):
return self.v1 + self.v2
def subtract(self):
return self.v1 - self.v2
def setV1(self, v):
if isinstance(v, int):
self.v1 = v
def getV1(self):
return self.v1
c = Cal(100, 10)
print(c.add())
print(c.multiply())
print(c.divide())
[참고한 자료]
'프로젝트 > 인공지능' 카테고리의 다른 글
[인공지능 #8]xor / xor-nn / xor-nn-wide-deep (0) | 2017.08.06 |
---|---|
[인공지능 #7] Rate overfiting , training/test data , nomalization (0) | 2017.08.05 |
[인공지능 #6 ] multinomial 적용 (0) | 2017.07.30 |
[인공지능 #5 ] Logistics classification 가설함수 (0) | 2017.07.30 |
[인공지능 #4 ] 여러개의 DATA를 (X 변수) 구현 (0) | 2017.07.30 |
[인공지능 #6 ] multinomial 적용
[ multinomial 적용]
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
# Lab 6 Softmax Classifier
import tensorflow as tf
tf.set_random_seed(777) # for reproducibility
x_data = [[1, 2, 1, 1],
[2, 1, 3, 2],
[3, 1, 3, 4],
[4, 1, 5, 5],
[1, 7, 5, 5],
[1, 2, 5, 6],
[1, 6, 6, 6],
[1, 7, 7, 7]]
y_data = [[0, 0, 1], ==>2
[0, 0, 1], ==>2
[0, 0, 1],
[0, 1, 0], ==>1
[0, 1, 0],
[0, 1, 0],
[1, 0, 0],
[1, 0, 0]]
X = tf.placeholder("float", [None, 4])
Y = tf.placeholder("float", [None, 3])
nb_classes = 3
W = tf.Variable(tf.random_normal([4, nb_classes]), name='weight')
b = tf.Variable(tf.random_normal([nb_classes]), name='bias')
# tf.nn.softmax computes softmax activations
==> 확율( 1~0 )으로 변환해줌
# softmax = exp(logits) / reduce_sum(exp(logits), dim)
hypothesis = tf.nn.softmax(tf.matmul(X, W) + b)
# Cross entropy cost/loss
cost = tf.reduce_mean(-tf.reduce_sum(Y * tf.log(hypothesis), axis=1))
D(S,L)=-SIGMA Li log(Si) 함수와
는 같은 결과를 나타냄
예측이 맞으면 COST는 작아지고, 틀리면 커지도록 구성한 함수임.
상기 함수(상기 2함수는 같은결과를 내고있음 , 같은 함수라고 할수있음)가 그 역할을 충실히 해내고 있음.
optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.1).minimize(cost)
COST 최소화 실행
# Launch graph
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
for step in range(2001):
sess.run(optimizer, feed_dict={X: x_data, Y: y_data})
if step % 200 == 0:
print(step, sess.run(cost, feed_dict={X: x_data, Y: y_data}))
print('--------------')
# Testing & One-hot encoding
a = sess.run(hypothesis, feed_dict={X: [[1, 11, 7, 9]]})
print(a, sess.run(tf.arg_max(a, 1)))
==> 예측치 출력(3개) , one hot 값 출력 ==> 어떤 클래스 인지를 결정해서 출력해줌
==> 예시)손글씨의 특징을 3개의 x 값으로 표현하고, 예측치를 출력 3개로 하며, 이때 확율이 가장높게나타난 클래스가 해당 손글씨가 됨.
==> 예시) 글래스(강아지,고양이...)등의 특징을 여러 x갋으로 표현하고, 예측을 y값으로 예측하고, 그중 확율이 제일 높은수치를 선택해줌, 제일 큰 수치가 해당클래스(고양이, 강아지 등)가 된다.
print('--------------')
b = sess.run(hypothesis, feed_dict={X: [[1, 3, 4, 3]]})
print(b, sess.run(tf.arg_max(b, 1)))
print('--------------')
c = sess.run(hypothesis, feed_dict={X: [[1, 1, 0, 1]]})
print(c, sess.run(tf.arg_max(c, 1)))
print('--------------')
all = sess.run(hypothesis, feed_dict={
X: [[1, 11, 7, 9], [1, 3, 4, 3], [1, 1, 0, 1]]})
print(all, sess.run(tf.arg_max(all, 1)))
'''
--------------
[[ 1.38904958e-03 9.98601854e-01 9.06129117e-06]] [1] ==> 전체의 합은 1 임
--------------
[[ 0.93119204 0.06290206 0.0059059 ]] [0]
--------------
[[ 1.27327668e-08 3.34112905e-04 9.99665856e-01]] [2]
--------------
[[ 1.38904958e-03 9.98601854e-01 9.06129117e-06]
[ 9.31192040e-01 6.29020557e-02 5.90589503e-03]
[ 1.27327668e-08 3.34112905e-04 9.99665856e-01]] [1 0 2]
'''
[ Fancy softmax classfication 구현 ]
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
# Lab 6 Softmax Classifier
import tensorflow as tf
import numpy as np
tf.set_random_seed(777) # for reproducibility
# Predicting animal type based on various features
xy = np.loadtxt('data-04-zoo.csv', delimiter=',', dtype=np.float32)
x_data = xy[:, 0:-1]
y_data = xy[:, [-1]]
print(x_data.shape, y_data.shape)
nb_classes = 7 # 0 ~ 6
X = tf.placeholder(tf.float32, [None, 16])
Y = tf.placeholder(tf.int32, [None, 1]) # 0 ~ 6
Y_one_hot = tf.one_hot(Y, nb_classes) # one hot
print("one_hot", Y_one_hot)
Y_one_hot = tf.reshape(Y_one_hot, [-1, nb_classes])
# one hot을 하면 rank가 1개 늘어나게 된다, 이를 다시 줄이는 역할을 하는것이 reshape 임
print("reshape", Y_one_hot)
W = tf.Variable(tf.random_normal([16, nb_classes]), name='weight')
b = tf.Variable(tf.random_normal([nb_classes]), name='bias')
# tf.nn.softmax computes softmax activations
# softmax = exp(logits) / reduce_sum(exp(logits), dim)
logits = tf.matmul(X, W) + b
hypothesis = tf.nn.softmax(logits)
# Cross entropy cost/loss
cost_i = tf.nn.softmax_cross_entropy_with_logits(logits=logits,
labels=Y_one_hot)
cost = tf.reduce_mean(cost_i)
optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.1).minimize(cost)
prediction = tf.argmax(hypothesis, 1)
correct_prediction = tf.equal(prediction, tf.argmax(Y_one_hot, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
# Launch graph
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
for step in range(2000):
sess.run(optimizer, feed_dict={X: x_data, Y: y_data})
if step % 100 == 0:
loss, acc = sess.run([cost, accuracy], feed_dict={
X: x_data, Y: y_data})
print("Step: {:5}\tLoss: {:.3f}\tAcc: {:.2%}".format(
step, loss, acc))
# Let's see if we can predict
pred = sess.run(prediction, feed_dict={X: x_data})
# y_data: (N,1) = flatten => (N, ) matches pred.shape
for p, y in zip(pred, y_data.flatten()):
print("[{}] Prediction: {} True Y: {}".format(p == int(y), p, int(y)))
'''
Step: 0 Loss: 5.106 Acc: 37.62%
Step: 100 Loss: 0.800 Acc: 79.21%
Step: 200 Loss: 0.486 Acc: 88.12%
Step: 300 Loss: 0.349 Acc: 90.10%
Step: 400 Loss: 0.272 Acc: 94.06%
Step: 500 Loss: 0.222 Acc: 95.05%
Step: 600 Loss: 0.187 Acc: 97.03%
Step: 700 Loss: 0.161 Acc: 97.03%
Step: 800 Loss: 0.140 Acc: 97.03%
Step: 900 Loss: 0.124 Acc: 97.03%
Step: 1000 Loss: 0.111 Acc: 97.03%
Step: 1100 Loss: 0.101 Acc: 99.01%
Step: 1200 Loss: 0.092 Acc: 100.00%
[참고자료 ]
https://www.inflearn.com/course/기본적인-머신러닝-딥러닝-강좌/
http://agiantmind.tistory.com/176
https://www.tensorflow.org/install/
https://github.com/hunkim/deeplearningzerotoall
http://www.derivative-calculator.net/
http://terms.naver.com/entry.nhn?docId=3350391&cid=58247&categoryId=58247 ==> 미분계산/공식
http://matplotlib.org/users/installing.html ==>matplotlib 설치
https://www.kaggle.com/==> DATA SAMPLE 을 구할수 있는 사이트
'프로젝트 > 인공지능' 카테고리의 다른 글
[인공지능 #7] Rate overfiting , training/test data , nomalization (0) | 2017.08.05 |
---|---|
[python] python 문법정리 (0) | 2017.08.05 |
[인공지능 #5 ] Logistics classification 가설함수 (0) | 2017.07.30 |
[인공지능 #4 ] 여러개의 DATA를 (X 변수) 구현 (0) | 2017.07.30 |
[인공지능 #3 ] Cost 최소화 알고리즘 구현 (0) | 2017.07.30 |