ML

Welcome back to the series on timeseries analysis. In this article, we’re going to discuss: plotting timeseries data and smoothing the data to handle outliers & make finding a trend a little bit easier. In the below, I have ingsted the timeseries data into a dataframe called df. From that dataframe, I have then set […]

The purpose of time series analysis is to analyse timeseries data and extrapolate the patterns you identify into the future, enabling us to make predictions. It’s important to understand when we can use timeseries analysis. If the timeseries has a clear pattern, trend and seasonality then you can accurately model a forecast. If a dataset […]

Pattern matching can be really very useful in large bodies of text. Yes, you could, for the most part, use a regular expression to achieve the same thing; but I tend to find the Spacy functionality a lot more straightforward – afterall, nobody enjoys the syntax of a regular expression. Further to this, Spacy makes […]

In this article, we are going to look into an unsupervised machine learning model, which aims to cluster documents into topics – the algorithm is called Latent Dirichlet Allocation (LDA). Essentially, we can assume that documents which are written about similar things, use a similar group of words. LDA tries to map all the documents […]

Text classification is all about classifying a body of text, without actually reading it. It’s a supervised machine learning algorithm which uses term frequency (how often words occur) to classify the document.  The classic example is to determine whether an email is spam or not. To do this, we can ingest a dataframe which includes […]

Recently, I have been presented with some problems which require Natural Language Processing to solve. Natural Language Processing (NLP) is a field of Artificial Intelligence which enables a machine to interpret human text for the purpose of analysis – for example, understanding the main topics discussed in a document or how positive customers reviews are […]

Data leakage is where we accidentally share data between the test and training sets; so,that our predictions are more accurate than they will ever be in the real world. If you’ve run your model against your test dataset and have a 99% accuracy, you probably have data leakage. Maybe you don’t, but you probably do. […]

Association rule mining is an unsupervised learning algorithm which finds patterns in our data, whereby we know how likely it is that Y occurs in the event of X. In other words, it finds features (or dimensions) which appear together frequently. Note: just because someone buys burgers with ketchup, does not mean someone that buys […]

For this article, I wanted to demonstrate an end to end model implementation, including tuning for an imbalanced dataset. First off, let’s ingest our data: Now, we can start to explore our dataset. First off, are there any null values that we need to worry about? There are no nulls! a rare pleasure. Let’s now […]

Tuning our machine learning model is of course critical to its accuracy. There are two methods for tuning models: tuning the data and tuning the parameters.  We ‘tune’ the data to clean it up, select the right features and make sure we don’t have a class imbalance – because afterall, if you pump garbage into […]

In the last article, we started to look at the implemented algorithm and looked at the model score along with the confusion matrix. In this article, I am going to cover off a high level overview of some of the methods we could use to evaluate our model performance. Classification Accuracy Classification Accuracy is the […]

Following on from my previous article; we are continuing our journey to predicting whether someone has diabetes. The way we do this is pretty straight forward. First, we get our data ready to be ingested by the model. To do this, we create two new dataframes y and X. The y dataframe stores the outcomes […]