Utilizing multi-core processor for classification in WEKA

Currently, WEKA is one of the most favorites machine learning tools. Without programming skills, you can do serious classification, clustering, and big data analysis. For some time I’ve been using its standard GUI features without thinking much about performance bottlenecks. But since researches are becoming more complex by using ensemble, voting and other meta-algorithms that generally are based on multiple classifiers running simultaneously, the performance issues start becoming annoying. You need to wait for hours until the task is completed. The problem is that when running classification algorithms from the WEKA GUI, the utilize a single core of your processor. Such algorithms as Multi-layer Percepron running 10 fold cross-validation is calculating one cross fold at the time on one core taking a long time to accomplish: So I started looking for options to make it use all cores of the processor as separate threads for each fold of operation. There…

Continue reading

Implementing logistic regression learner with python

Logistic regression is the next step from linear regression. The most real life data have a non-linear relationship, thus applying linear models might be ineffective. Logistic regression is capable of handling non-linear effects in prediction tasks. You can think of lots of different scenarios where logistic regression could be applied. There can be financial, demographic, health, weather and other data where the model could be implemented and used to predict next events on future data. For instance, you can classify emails into span and non-spam, transactions being fraud or non, tumors being malignant or benign. In order to understand logistic regression, let’s cover some basics, do a simple classification on data set with two features and then test it on real life data with multiple features.

Continue reading

Future of Artificial Intelligence; Its Implementations and Limits

Artificial Intelligence (AI) is the field of computer science which uses mechanical and computational processes to echo almost all aspects of human intelligence. AI can perform multiple functions: sensory interaction with the environment and the ability to make decisions about events that haven’t happened yet without any human assistance whatsoever. Targeted advertising and virtual agents that recognize the patterns of your behavior are much standards in today’s online undertakings. Artificial Intelligence is used by business enterprises in data analysis algorithms which have the highest advantage of analyzing the Big Data, and it also involves customer engaging techniques. Apart from IBM which developed some of the earliest functions of AI, Google and Facebook are also using AI for the analytic purpose of the massive amount of data they receive.

Continue reading

Building and evaluating Naive Bayes classifier with WEKA

This is a followup post from previous where we were calculating Naive Bayes prediction on the given data set. This time I want to demonstrate how all this can be implemented using WEKA application. For those who don’t know what WEKA is I highly recommend visiting their website and getting the latest release. It is a compelling machine learning software written in Java. You can find plenty of tutorials on youtube on how to get started with WEKA. So I won’t get into details. I’m sure you’ll be able to follow anyway.

Continue reading

Simple explanation of Naive Bayes classifier

Probably you’ve heard about Naive Bayes classifier and likely used in some GUI based classifiers like WEKA package. This is a number one algorithm used to see the initial results of classification. Sometimes surprisingly it outperforms the other models with speed, accuracy and simplicity. Lets see how this algorithm looks and what does it do. As you may know algorithm works on Bayes theorem of probability which allows to predict the class of unknown data set. Hoe you are comfortable with probability math – at least some basics.

Continue reading

Linear regression with multiple features

linear regression cost function

Single feature linear regression is simple. All you need is to find a function that fits training data best. It is also easy to plot data and learning curve. But in reality regression analysis is based on multiple features. So in most cases, we cannot imagine the multidimensional space where data could be plotted. We need to rely on the methods we use. You have to feel comfortable with linear algebra where matrices and vectors are used. If previously we had one feature (temperature) now we need to introduce more of them. So we need to expand hypothesis to accept more features. From now and later on instead of output y we are going to use h(x) notation: As you can see with more variables (features), we also end up with more parameters θ that has to be learned. Before we move let’s find suitable data that we could use…

Continue reading

Linear regression – learning algorithm with Python

In this post, we are going to demystify the learning algorithm of linear regression. We are going to analyze the simplest univariate case with single feature X wherein the previous example was temperature and output was cricket chirps/sec. Let’s use the same data with crickets to build learning algorithm and see if it produces a similar hypothesis as in excel. As you may already know from this example, we need to find linear equation parameters θ0 and θ1, to fit line most optimally on the given data set: y = θ0 + θ1 x x here is a feature (temperature) and y – output value (chirps/sec). So how we are going to find parameters θ0 and θ1? The whole point of the learning algorithm is doing this iteratively. We need to find optimal θ0, θ1 parameter values, so that approximation line error from the plotted training set is minimal. By…

Continue reading