Perceptron Challenge By: Colleen McGuckin

              After our first meeting, we learned the basics of perceptrons. This challenge had us manipulate the original code from our first meeting, to make it so that the code had three features, could handle three dimensional data, and different numbers of training and testing data. This inherently led to our creating of 3D graphs, as depicted above. This first challenge was very fun, and I guarantee you will have fun completing it.

Perceptron Code Challenge - Alyssa Koo

After learning the basics of the perceptron concepts a few weeks ago, we put our skills to the test and used a new set of data, and was able to create a 3D graph that displayed our data, to the equation that we got from utilizing the training data. The code was able to pick up the pattern, and create an equation that was the best line of fit for the given data. 

    Perceptron Code

Perceptron Code Challenge - Emma Ruzicka

This was fun!!! We trained and tested a set of data with three different variables. Thus, naturally, plotting it on a 3D graph would be the most efficient and visually understandable. We plotted the testing data to see the patterns and it seems to have worked pretty well! :)

The key was simply to adjust the code to be able to include three types of data: 3D Perceptron Code

Machine Learning Startups and Applications

Machine Learning is no longer a fancy sci-fi dream or an obscure research field in the dark corner of Google, Facebook, Uber and Baidu have acquired most of the dominant players in this space to improve their services. The most well-known one is that Google spent over $500 million for a little-known London startup called DeepMind in 2014! Many small startups are trying eagerly to find a niche in this new game field. Since machine learning can have huge impacts to various fields in our society, we would like to explore its real-world applications while we are picking up the technical know-how. I am attaching the links of two online articles on this topic. I am looking for some volunteers to summarize the info in these articles into excel/spreadsheet formats, and share the links of them on our blog. These can be working documents that we can keep adding new information. This efforts will help us proceed to the next stage of research. At the same time, you can start thinking about what types of real-world problems in your life can potentially be solved by machine learning techniques. You can share your thoughts with us on blog. 

the ivory tower. It is currently a highly competitive research field for both academia and industry. Industry giants such as

• 10 Cool Machine Learning Startups To Watch
• 11 Cool Ways to Use Machine Learning

AI Workshop I and Perceptron Challenge

Girls hanging out after the AI workshop I

WORKSHOP 

 

During the AI Workshop I today, nine students including most of our research group members have been trained to understand perceptron from concepts to code-level implementation (in C language)! Thanks to Elro seniors, Qin Ying and Tiffany, presented the key perceptron concepts to us! It is our first contact with one of the most important machine learning technique- Artificial Neural Network (ANN). The perceptron is an algorithm for supervised learning. It is a type of linear classifier which can classify linearly separable data. This basic training will form the foundation for the next-level neural network- Multi-layer Perceptron (MLP).

In case you missed the workshop, or you want to review the perceptron concept in more details, you can watch the following video tutorials:

• How the Perceptron Algorithm Works 1/2  
• How the Perceptron Algorithm Works 2/2

CHALLENGE

As you finish the perceptron training, you have a new challenge to tackle - modifying the

perceptron code to classify a new data set. The information of the data set is listed in the following:

1. No.of classes: 2 (class 0 and 1)
2. No. of training data: 2000
3. No. of testing data: 400
4. No. of features: 3 (feature vector is in 3-dimensional space)
5. Files: training.txt, and testing.txt

Since the original perceptron is only for 2-dimensional data, you need to identify and modify all the dimension-related code to make it work. After you finishing the classification, visualize your training and testing data, and the decision boundary using Grapher in Mac laptop (see example). Our deadline for this challenge is March 1, 2017 (Wednesday). You can post your results onto the blog once you finish the project. Please feel free to drop by my room (501) for any questions. If you need computer resource to work on it during the break, please contact me in advance.

Workshop Material: Perceptron Training and Classifying Code

Please click the perceptron code link to download the code that includes training and classification.

There are two project folders in the zip file:

1. PerceptronTrainer: Perform training based on training data in training.txt and save the trained weight at weights.txt.
2. PerceptronClassifier: Perform classification based on the weights.txt and output the result to classification.txt.

You have to set the schemes (working directories) of both projects to the PerceptronTrainer folder where the weights.txt file is located. So, they can share the file at the same location.

AI Workshop 1: Artificial Neural Network - Perceptron

AI Workshop 1: Artificial Neural Network - Perceptrons

Hi All,

We are planning to have our first AI workshop some time next week. The goals are learning the basics of Neural Network. The topics include:

• Neurons & Neural Network
• Artificial Neurons
• Basic operations of Perceptron
• Decision Boundary
• Learning Rules of Perceptron
• Limitations of Perceptron

During the workshop, we are going to implement a Single-Layer Perceptron (SLP) in C language, and see how it help us solve real-world problems. The workshop will be ~2 hours long. You are welcome to invite your friends.

In order to facilitate the meeting, please use the survey form to indicate your availability. I will let you know the date as soon as the group fill it out. We will make the final decision based on the survey results.

Introduction to Machine Learning Course

I just found this Machine Learning (ML) Course, Introduction to Machine Learning, on YouTube! It is a course taught by Professor Alex Ihler at University of California, Irvine. I watched a few tutorials and they seem not too hard for high school students to understand. There are 35 short video tutorials (most of them < 20 minutes) covering wide range topics in ML. I suggest that you can watch 2-3 tutorials every week to gradually build up the foundation in ML. We are going to zoom into a very specific field of ML called "Neural Network" in the next few weeks. Following through this course can give you a broader foundation and perspective for ML. You are encouraged to post your notes and questions when you watch these videos. Enjoy!

Notes: Machine Learning

30 minutes of continuous information is hard to absorb... So, to summarize "A Friendly Introduction to Machine Learning", here are the definitions:
Feel free to read these as you watch the video, they go in order as they are explained in the video and may help define what is going on as it is being visually represented.

• Machine Learning: The ability of a computer to learn like a human, from experience. For computers, however, this experience, is substituted by data that it can analyze, store, and compare. 
• Linear Regression: Comparison of data on a coordinate plane; Data points are marked on a graph and a line that best fits the data is derived, so that it has the least amount of error. The error is measured by the distance of all the points from the line: The greater the sum of the distances, the greater the error; This method also works with polynomials, parabolas, etc. 
• Gradient Descent: an algorithm used to minimize functions; A continuous procedure that adjusts a function, in order to provide the most optimal result or decrease the error to find its minimum value; Uses calculus!
• Naive Bayes: Providing solutions based on probability; Probable characteristics for a desired output are evaluated. The inputs with the most of these characteristics are the first to be considered.
• Decision Trees: Comparison of data based on a table; Using multiple features to split the data continuously to narrow it down to individual users/outputs. (You can think of it as a series of if statements that keep splitting the [remaining] data into two categories for x features or characteristics)
• Logistic Regression: Comparison of data on a coordinate grid; Looking for previous trends to divide data. Like Linear Regression, a line is used to divide the points. Except, the error is measured by the number of points that are wrongly classified by the division of the graph with the line, based on a given condition (ex.: pass or fail).
• Gradient Descent is, again, used to minimize the error.
• Neural Networks: Comparison of data on a coordinate grid; Using multiple lines to split the data based on multiple conditions. Like an and-statement, the data that fits all conditions is our output; A combination of multiple logistic regression graphs.
• Support Vector Machines: Finding a line that maximizes the distance from boundary points through linear optimization; Separating the data into two sections and split it evenly between them. (Think of this kind of like finding the middle of a middle).
• Kernel Trick: Helps support Vector Machines; Finding a function that splits the data based on certain set mathematical properties and similarity between the data points; As a mapping function, it can often be perceived as working in a 3D space. The additional dimension serves as a way to split the data accordingly. 
• Clustering: Grouping data points based on proximity until a distance limit is met; Essentially, this means grouping data points with similar values together to classify the data. By determining the limit between the differences of the values, one can manipulate how many or how large the groups are. (hierarchical or K-means)

Tutorial: A Friendly Introduction to Machine Learning

I just found this nice Machine Learning introductory video. It explains and illustrates a few most popular machine learning techniques in an extremely friendly way! The topics include Linear and Polynomial Regression, Gradient Descent,  Naive Bayes, Decision Tree, Logistic Regression, Neural Network, Support Vector Machine and Kernel Trick. This video tutorial can help you grasp the basic concepts and possible applications of machine learning quickly. You are encouraged to go through it a few times and take notes as you watch, and find further resource online to go deeper in any topics interested you. You can share your notes and finding with us by posting onto this blog. The expected time to finish this tutorial is by February 5 (Sunday).

World's Youngest IBM Watson Programmer

It is fun and informative to watch this young gentleman explaining his Natural Language Q&A (NLQA) system. Enjoy it!