Neural Nets 2: Difference between revisions

Latest revision as of 15:59, 22 January 2018

TUTORIAL ON MACHINE LEARNING AND NEURAL NETWORKS (PAGE 2)

Tutorial Pages

1. Intro
2. Network Inputs
3. Network Activation
4. Network Outputs

The previous page ended by mentioning that neural net classifiers can fail for two independent reasons:

The neural network itself is ill-formulated, so no matter how much information is provided it cannot learn to solve the classification problem.
The input data is insufficient in quantity or format to solve the classification problem.

That said, we found that a single neuron with input from a single feature was sufficient to solve the previous classification problem. Let's try another one. Without changing anything yet, press the play button on the Tensorflow playground below and let it run for about 5 seconds, and then press pause...

Tensorflow

Finished?

How'd it do on this new classification task? Is one neuron with input from a single feature (the dim1 data: AGE) still performing well to crate a background color gradient that matches the color of the dots? I'm assuming it didn't do so well this time around. Try again. Resent Tensorflow, and more neurons, and an additional layer also several neurons. Rerun.

How'd it do? It still can't solve the classification problem right? It seems like we'll need to add an additional feature as input. Let's take a look at our options. Again, here is the full set of possible network inputs...

Input Features
X₁	AGE
X₂	SCORE
X₁²	AGE²
X₂²	SCORE²
X₁X₂	AGE × SCORE
sin(X₁)	sin(AGE)
sin(X₂)	sin(SCORE)

Start by running X₁ and X₂ together, just to see what happens. Then feel free to test out whatever combinations you think might work.

Input Combinations

DISCUSSION TBD

Neuron Activation Function

The output function from each neuron in our neural net was chosen to be the tanh function.

The tanh function is an extremely common choice for an output function in artificial neural network machine learning frameworks because it yields a nice sigmoid shape, and no matter the magnitude of its inputs, the output from the tanh function is bounded between { 0 : 1}. These are very desirable properties for neural net nodes. Here you see the tanh function evaluated across various x-dim inputs...

Error creating thumbnail: File missing

see tanh on wolfram alpha for many details about tanh function.

Tanh produces a sigmoid output over the range {-2 : 2}, and automatically evaluates to exact values when its argument is the natural logarithm. Speaking of the natural log, that is another very common choice of output function for the same reasons as tanh.

This is something to keep this in mind, if you're wondering what sorts of numbers are travelling along the axons of these neurons, and ultimately those colored gradients underneath the dots.

@@ Line 5: / Line 5: @@
 . [[Neural Nets 4|Network Outputs]]<br>
 }}
-<br><br><br><br><br><br>
+<br>
-<br><br><br><br><br><br>
-==TBD BELOW==
+The previous page ended by mentioning that neural net classifiers can fail for two independent reasons:
-Below I've embedded another Tensorflow neural net playground.
+# The neural network itself is ill-formulated, so no matter how much information is provided it cannot learn to solve the classification problem.
+# The input data is insufficient in quantity or format to solve the classification problem.
-{{#widget:Tensorflow}}
+<br>
+That said, we found that a single neuron with input from a single feature was sufficient to solve the previous classification problem. Let's try another one. Without changing anything yet, press the play button on the Tensorflow playground below and let it run for about 5 seconds, and then press pause...
+===Tensorflow===
+----
+{{#widget:Tensorflow2}}
+<br><br>
+====''Finished?''====
+How'd it do on this new classification task? Is one neuron with input from ''a single feature'' (the dim1 data: AGE) still performing well to crate a background color gradient that matches the color of the dots? I'm assuming it didn't do so well this time around. Try again. Resent Tensorflow, and more neurons, and an additional layer also several neurons. Rerun.
+How'd it do? It still can't solve the classification problem right? It seems like we'll need to add an additional feature as input. Let's take a look at our options. Again, here is the full set of possible network inputs...
+{| class="wikitable" width=30% align=center
+|+ style="font-weight:bold;"|Input Features
+|- style="height:30px"
+| style="background:#f7f7f7; border:3px solid #ffffff"| ''X''<sub>1</sub>
+|colspan=2 style="background:#f7f7f7; border:3px solid #ffffff"| AGE
+|- style="height:30px"
+| style="background:#f7f7f7; border:3px solid #ffffff"| ''X''<sub>2</sub>
+|colspan=2 style="background:#f7f7f7; border:3px solid #ffffff"|  SCORE
+|- style="height:30px"
+| style="background:#f7f7f7; border:3px solid #ffffff"| ''X''<sub>1</sub><sup>2</sup>
+|colspan=2 style="background:#f7f7f7; border:3px solid #ffffff"|  AGE<sup>2</sup>
+|- style="height:30px"
+| style="background:#f7f7f7; border:3px solid #ffffff"| ''X''<sub>2</sub><sup>2</sup>
+|colspan=2 style="background:#f7f7f7; border:3px solid #ffffff"|  SCORE<sup>2</sup>
+|- style="height:30px"
+| style="background:#f7f7f7; border:3px solid #ffffff"| ''X''<sub>1</sub>''X''<sub>2</sub>
+|colspan=2 style="background:#f7f7f7; border:3px solid #ffffff"|  AGE × SCORE
+|- style="height:30px"
+| style="background:#f7f7f7; border:3px solid #ffffff"| sin(''X''<sub>1</sub>)
+|colspan=2 style="background:#f7f7f7; border:3px solid #ffffff"|  sin(AGE)
+|- style="height:30px"
+| style="background:#f7f7f7; border:3px solid #ffffff"| sin(''X''<sub>2</sub>)
+|colspan=2 style="background:#f7f7f7; border:3px solid #ffffff"|  sin(SCORE)
+|}
+Start by running ''X''<sub>1</sub> and  ''X''<sub>2</sub> together, just to see what happens. Then feel free to test out whatever combinations you think might work.
+===Input Combinations===
+----
+====''DISCUSSION TBD''====
-<!-- [[File: Tensorflow Tutorial img1.png]] -->
-{{Clear}}
-<br><br><br>
-===Outputs===
+===Neuron Activation Function===
 ----
-More directly, it is the value spit-out by the activation function of the 'output layer'. Here, since we only have a single layer, our hidden 'hidden layer' and 'output layer' are one in the same. The output function of our neuron is known as the '''tanh''' function.
+The output function from each neuron in our neural net was chosen to be the '''tanh''' function.
 The tanh function is an extremely common choice for an output function in artificial neural network machine learning frameworks because it yields a nice sigmoid shape, and no matter the magnitude of its inputs, the output from the tanh function is bounded between { 0 : 1}. These are very desirable properties for neural net nodes. Here you see the tanh function evaluated across various x-dim inputs...
@@ Line 32: / Line 75: @@
 <br><br><br><br>
 {{Clear}}
-[[File: Tanh.png|thumb|500px|left|see [http://reference.wolfram.com/language/ref/Tanh.html tanh on wolfram alpha] for many details about tanh function.]]
+[[File: Tanh.png|thumb|400px|left|see [http://reference.wolfram.com/language/ref/Tanh.html tanh on wolfram alpha] for many details about tanh function.]]
 {{Clear}}
 Tanh produces a sigmoid output over the range {-2 : 2}, and automatically evaluates to exact values when its argument is the natural logarithm. Speaking of the natural log, that is another very common choice of output function for the same reasons as tanh.
-For now, let's not belabor the point that our neuron (and in going forward, all our neurons) are using the tanh function. Maybe just keep this in mind if you're wondering what sorts of numbers are travelling along the axons of these neurons, and ultimately those colored gradients underneath the dots.
+This is something to keep this in mind, if you're wondering what sorts of numbers are travelling along the axons of these neurons, and ultimately those colored gradients underneath the dots.
+====''TO BE CONTINUED...''====
+<br>
+<!-- {{SmallBox|'''[[Neural Nets 2|Continue to Neural Nets Tutorial Page 2]]'''}} -->
+<!-- <btn data-toggle="tooltip">Neural Nets 2</btn> -->
-This tutorial continues on the next page. Don't worry about playing around too much with the TensorFlow GUI, there will be plenty of that on the next page, and those that follow.
+__NOTOC__