Kodey | Data Science | Data Engineering | Machine Learning
Data Engineering & Data Science For Everyone | Python Code | PySpark | Machine Learning | Statistics | Predictive Analytics | Data VIsualization | Time Series Analysis | NLP
This article will overview how to extract data through screen scraping from a website. Specifically, this will focus on the UK government, open data website. Scraping websites is a contentious topic – while some websites don’t mind you doing it; some really would rather you didn’t and put in measures to try and stop you. […]
Read more
Let’s say, we’re working for a retailer. They have two distribution centres in Australia – the geographic area covered by each distribution centre is coloured in orange. Our task is to, from this image, work out to how many square kilometers of Australia can we deliver and what percentage of the country is that? Right, […]
Read more
As data engineers and data scientists, we’re spend a lot of time exploring data. When you’re working with huge datasets, you may find you need to utilise Apache Spark or similar to conduct this exploratative analysis but for the majority of use-cases, Pandas is the defaqto tool we choose. The Pandas library has been so […]
Read more
Another GUI for Pandas! YES – they’re coming out of the woodwork now! But this one is a little bit different to the Pandas GUI library I discussed previously. Pandas GUI let us slice and dise our data; restructure it and visualize it. Pandas Profiling isn’t as feature-rich but provides a different way of looking […]
Read more
Pandas is the defaqto library for data analysis in Python for good reason. It’s infinitely flexible, relatively performant and easy to learn. That said, quite a lot of what we, as data scientists and engineers do is trial and error and exploration. This exploratative process can sometimes be a bit tedius. We create a dataframe; […]
Read more
As you may be aware, I am a Python tutor online and quite often I get asked pretty specific questions. This week, I was asked to show a simple way to find the distance in kilometres between two geographic locations. So, here it is then – the easiest way to approach this problem is to […]
Read more
The below script takes the input of a UK postcode and ensures that it matches a valid format. I have handled the below formats: X11XX XX11XX XX111XX To do this, I use some regular expressions. Let’s look at one: “^[a-zA-Z]{1}[0-9]{2}[a-zA-Z]{2}”. Here: The beginning of the string (denoted with a ^) needs to be a letter. […]
Read more
Today I thought it might be cool to make a super simple little text based game in Python in my spare 15 minutes. So I made this hangman game. As you can see, the word chosen is ‘squiggle’. As the user selects new letters, all occurences of those letters are uncovered in the list. If […]
Read more
Often, you will want to format your numbers in Python because it’s quite hard for a user to read 10000000000 and immediately understand it as: 10,000,000,000. So, this little article will show you one possible way to do the formatting – using many of the Python concepts discussed in previous articles and in my Udemy […]
Read more
The Python zip function is incredibly useful. It takes in an iterable (a list, a tuple, a dictionary etc..) and returns an iterator. Well, that’s as clear as mud, right? Let’s look at an example to clear things up. Let’s imagine I have three lists like below. These lists are related to one another. Index […]
Read more
Those of you that know me will know I am not a fan of Lambda functions. I just don’t buy into the notion of making code as concise as possible at the detriment of readability. To show you what I mean, I have written two functions. One is a Lambda function and the other is […]
Read more
This is a really quick post about filtering in Python. In the below, you will see that we have a list of animals. I want to filter that list based on a boolean condition – the name of the animal starts with the letter C: True or False. The resulting filter will return a list […]
Read more