Intro to Tensorflow
rpi.analyticsdojo.com
Adopted from Hands-On Machine Learning with Scikit-Learn and TensorFlow by Aurélien Géron.
Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
For full license see repository.
!pip install tensorflow
Requirement already satisfied: tensorflow in /anaconda3/envs/carme/lib/python3.6/site-packages (1.0.0)
Requirement already satisfied: wheel>=0.26 in /anaconda3/envs/carme/lib/python3.6/site-packages (from tensorflow) (0.31.0)
Requirement already satisfied: numpy>=1.11.0 in /anaconda3/envs/carme/lib/python3.6/site-packages (from tensorflow) (1.12.1)
Requirement already satisfied: six>=1.10.0 in /anaconda3/envs/carme/lib/python3.6/site-packages (from tensorflow) (1.11.0)
Requirement already satisfied: protobuf>=3.1.0 in /anaconda3/envs/carme/lib/python3.6/site-packages (from tensorflow) (3.5.1)
Requirement already satisfied: setuptools in /anaconda3/envs/carme/lib/python3.6/site-packages (from protobuf>=3.1.0->tensorflow) (39.0.1)
kaggle-cli 0.12.13 has requirement lxml<4.1,>=4.0.0, but you'll have lxml 3.8.0 which is incompatible.
awscli 1.14.32 has requirement botocore==1.8.36, but you'll have botocore 1.9.7 which is incompatible.
apache-airflow 1.9.0 has requirement bleach==2.1.2, but you'll have bleach 2.1.3 which is incompatible.
apache-airflow 1.9.0 has requirement flask<0.12,>=0.11, but you'll have flask 0.12.2 which is incompatible.
apache-airflow 1.9.0 has requirement jinja2<2.9.0,>=2.7.3, but you'll have jinja2 2.10 which is incompatible.
# Common imports
import numpy as np
import os
# to make this notebook's output stable across runs
def reset_graph(seed=42):
tf.reset_default_graph()
tf.set_random_seed(seed)
np.random.seed(seed)
82. Tensorflow Graph Creation#
Tensorflow at it’s base is trying to enable computation.
Google designed tensorflow to scale, working with CPU, GPU, TPU
Tensorflow separates the graph into a construction component and a computational component
import tensorflow as tf
reset_graph()
x = tf.Variable(3, name="x")
y = tf.Variable(4, name="y")
f = x*x*y + y + 2
f
<tf.Tensor 'add_1:0' shape=() dtype=int32>
83. Tensorflow Graph Computation#
Two different syntax
Similar to other roles.
# This is one way of creating a session.
sess = tf.Session()
sess.run(x.initializer)
sess.run(y.initializer)
result = sess.run(f)
sess.close()
print(result)
42
#This is like creating a file, we don't have to open and close.
with tf.Session() as sess:
x.initializer.run()
y.initializer.run()
result = f.eval()
print(result)
42
#This is like creating a file, we don't have to open and close.
init = tf.global_variables_initializer()
with tf.Session() as sess:
init.run()
result = f.eval()
print(result)
42
84. Managing Default Graphs#
The default graph is the one that will be computed.
reset_graph()
x1 = tf.Variable(1)
x1.graph is tf.get_default_graph()
True
graph = tf.Graph()
with graph.as_default():
x2 = tf.Variable(2)
x2.graph is graph
True
x2.graph is tf.get_default_graph()
False
85. Linear Regression#
85.1. Regression Using the Normal Equation#
The bias node in a neural network is a node that is always ‘on’. That is, its value is set to 1 without regard for the data in a given pattern. It is analogous to the intercept in a regression model, and serves the same function.
You can read more about the normal equation here.
import numpy as np
from sklearn.datasets import fetch_california_housing
#Reset the graph
reset_graph()
#Get the data
housing = fetch_california_housing()
m, n = housing.data.shape
#add bias term
housing_data_plus_bias = np.c_[np.ones((m, 1)), housing.data]
housing_data_plus_bias
array([[ 1. , 8.3252 , 41. , ..., 2.55555556,
37.88 , -122.23 ],
[ 1. , 8.3014 , 21. , ..., 2.10984183,
37.86 , -122.22 ],
[ 1. , 7.2574 , 52. , ..., 2.80225989,
37.85 , -122.24 ],
...,
[ 1. , 1.7 , 17. , ..., 2.3256351 ,
39.43 , -121.22 ],
[ 1. , 1.8672 , 18. , ..., 2.12320917,
39.43 , -121.32 ],
[ 1. , 2.3886 , 16. , ..., 2.61698113,
39.37 , -121.24 ]])
#Do the math with Tensorflow.
X = tf.constant(housing_data_plus_bias, dtype=tf.float32, name="X")
y = tf.constant(housing.target.reshape(-1, 1), dtype=tf.float32, name="y")
XT = tf.transpose(X)
theta = tf.matmul(tf.matmul(tf.matrix_inverse(tf.matmul(XT, X)), XT), y)
with tf.Session() as sess:
theta_tensorflow = theta.eval()
print(theta_tensorflow)
[[ -3.74651413e+01]
[ 4.35734153e-01]
[ 9.33829229e-03]
[ -1.06622010e-01]
[ 6.44106984e-01]
[ -4.25131839e-06]
[ -3.77322501e-03]
[ -4.26648885e-01]
[ -4.40514028e-01]]
Compare with pure NumPy
X = housing_data_plus_bias
y = housing.target.reshape(-1, 1)
theta_numpy = np.linalg.inv(X.T.dot(X)).dot(X.T).dot(y)
print(theta_numpy)
[[ -3.69419202e+01]
[ 4.36693293e-01]
[ 9.43577803e-03]
[ -1.07322041e-01]
[ 6.45065694e-01]
[ -3.97638942e-06]
[ -3.78654265e-03]
[ -4.21314378e-01]
[ -4.34513755e-01]]
Compare with Scikit-Learn
from sklearn.linear_model import LinearRegression
lin_reg = LinearRegression()
lin_reg.fit(housing.data, housing.target.reshape(-1, 1))
print(np.r_[lin_reg.intercept_.reshape(-1, 1), lin_reg.coef_.T])
[[ -3.69419202e+01]
[ 4.36693293e-01]
[ 9.43577803e-03]
[ -1.07322041e-01]
[ 6.45065694e-01]
[ -3.97638942e-06]
[ -3.78654265e-03]
[ -4.21314378e-01]
[ -4.34513755e-01]]