Posts

What is a TIL post? TIL - Stands for today I learnt. These posts will be shorts summaries of something I learnt “today”. The TIL posts have a low barrier to entry and are a lot easier to write. Therefore they should encourage me to post more often. In all my blog posts I try to post about things that I struggled to do. If I struggled to implement a feature or build something. It shows that there wasn’t a tutorial or an easy example I could follow. So then I usually decide to write my own, to make it easier for others. It also has a nice side effect of helping me remember the topic better. ...

Previous article This article assumes you are familiar with dotfiles and Dobot. If you want to know more about Dotbot click here In this article I will show you how you can use Dotbot plugins. We can use Dotbot plugins to run new directives such as apt. So we can use the apt package manager, so install packages. Tip One useful use case is when we setup on a new system we may want to make we have some packages installed like vim or make. We can automate some of this with Dotbot and its plugins. ...

Previous article This article assumes you are familiar with dotfiles and Dobot. If you want to know more about Dotbot click here If you are like me you have devices, such as personal desktop and a work laptop. These devices share some dotfiles but also have specific program and applications. For example: Personal Desktop: Linux Work Laptop: Windows (WSL) On Linux I use alacritty and in Windows I use Windows terminal I could copy all my dotfiles over and not worry about which programs exist on which systems. ...

Today I will go over my current VSCode setup covering both extensions and settings. Out of date This article may be out of date, by the time you are reading this. It is accurate as of 16th October 2022. Extensions Here we will go over some extensions I find very useful: Full list of my plugins here Better Comments Link: Better Comments As the name implies it improves your comments. Makes them look nicer if you add certain annotations. Such as adding TODO to the comment will turn the comment orange. See the image below: ...

If you’re like me you find yourself moving between multiple systems. Whether that be between my personal desktop and my work laptop or distro hopping on Linux. See relevant meme below: What are dotfiles? Many tools/program store their configuration files as files on your machine. On Linux you will often find these in ~/.config directory. Some common examples of dotfiles: - .vimrc - .bashrc - .gitconfig I wanted to find an easy way to manage my dotfiles and share them between mutiple systems. I also wanted a easy way to install software/tools I used between my systems. Introducing DotBot, a tool that provides an easy way to manage our dotfiles using VCS (git). ...

Shameless plug: This is related to a EuroPython 2022 talk I am giving, My Journey Using Docker as a Development Tool. For most of my common dev tasks, I’ve started to rely on docker/docker compose to run commands locally. I have also started using vscode’s .devcontainers, to provide a consistent environment for all developers using a project. The main reason for this is to avoid needing to install dependencies on my host machine. In theory, all I should need is a Docker daemon and a CLI (docker CLI) to interact with that Daemon. This also makes it far easier for any new developer to start working on my project and get set up. ...

In this article I will show you how you can have separate modules for your Socketio event handlers. Rather than keeping them all in the same file. Hopefully this should be a relatively short article, lets get into it Main In this example I will be using SocketIO alongside FastAPI, but you can easily change this code to be SocketIO only. I also will be using a uvicorn to run the server. ...

Background This will be a slightly shorter article. In this article I will show you how I’ve managed to do some end-to-end testing with Gitlab CI services. I’m building a browser-based multiplayer game called Banter Bus. Banter Bus consists of three main components, gui: A SvelteKit based frontend the user will interact with to play the game core-api: A Socketio API written in Python management-api: A simple RESTful API written in Python (FastAPI) Now say I want to write some e2e Cypress tests, that will test all of these components interacting with each other. Which mainly will look something like gui -> core-api -> management-api. ...

In this article I will show you how you can test an async Socketio application in Python, where the ASGI server we are running is uvicorn. I will be referring to these tests as integration tests, though depending on who you ask they could be called E2E tests, system tests, slow test etc. What I am referring to is simply testing out the entire “flow” of a socketio event i.e. emitting an event from a client, then receiving it on the web service and for my actual projects interacting with an actual database. ...

In this blog post, we will go over how we can automatically test a Gatsby site end-to-end (e2e), using Cypress on Gitlab CI. Introduction Gatsby Gatsby is a static site generator (SSG) built upon React. It allows us to create “blazing” fast websites. In this example, we will use a simple blog starter template available and add a Cypress test. Cypress Fast, easy and reliable testing for anything that runs in a browser. - Cypress README ...

Background A bit of background before we start the article. When I develop a Python web service I use the Connexion library created by Zalando. It’s a great library which is built on top of Flask. It uses an OpenAPI Specification (OAS) file to handle input validation and routing for you. Therefore reducing the boilerplate code you need to write. The main advantage of this is that we have a design-first approach to developing our API. We fully define the OAS then develop the code/web service. This also keeps the OAS up to date, helping to mitigate the issue of the code/documentation getting out of date. Especially when you share the OAS with other people (clients) to use. The last thing you want to do is give them an out-of-date file. ...

Recently I’ve been working on a new personal project called Banter Bus, a browser-based multiplayer game. I’ve been working on a REST API to add new questions to the game. The API is built in Golang and uses MongoDB as the database. Since Golang is a strongly typed language, we will need to specify the structure of the data we expect from the database. This can get tricky if the data varies, such as one field changing. ...

In this article, we will go over what the yield keyword is used for. We will also cover how you can use a yield with a pytest fixture to allow us to “teardown” tests, after all of our tests have run. A common job being removing test data from the database, so that next time your run the tests your tests won’t fail. Due to the database being in a different (unexpected) state. ...

A lot of people, I included, are using Gatsby to build their own blogs. One of the things I wanted to add to my blog was a table of contents (ToC) 📝. A ToC will show you all the headings of an article and when you click on a heading it’ll take you directly to that heading. It’s a nice little feature to have on your blog, which makes it easier for users to navigate and find the information they are looking for. ...

In this article, we will briefly go over what DNS (domain name system) is and explain how it is used in conjunction with Docker 🐳. DNS You can think of DNS like a phonebook, except instead of people’s name and phone numbers, it stores domains names and IP addresses (this can be either IPv4 or IPv6). Where a domain name is used to identify resources i.e. google.com is a domain name. This is how DNS works: ...

In this article, we will go over how you can use proxychains to proxy our traffic through a socks proxy. Background Recently, like everyone else, I’ve been working from home a lot more often. This means to access resources at work I need to use a VPN. However, to access some resources, such as production servers from my local machine, I need to use a SOCKS5 proxy. Without using a SOCKS proxy, I would need to do something shown in the diagram below. ...

In this article, we will go over how you can manage your markdown blog posts from another git repository (repo). Separate to the git repository for your Gatsby site. This is the same process that I use to manage this repo. So what this entails is the source code for my Gatsby site is in a repo called portfolio-site on Gitlab. Then I have another repo for all of my blog posts (in markdown) called articles. During build time of the Gatsby blog, we will import the markdown files from our articles git repo and use it as a source of data for our Gatsby blog. ...

In this article, we will look at how we can add an “edit post” button, to your Gatsby blog. When this button is clicked it will take the user to your markdown file, on github/gitlab that was used to generate the blog post they are currently viewing. Setup Before we add the edit button to a Gatsby blog, let’s set up a simple Gatsby site using the Gatsby blog starter. You can skip this step and add the button to an existing site. ...

In this article, we will go over how we can use the gitlab-auto-mr CLI script I wrote to help automate your Gitlab workflow. This is a very simple script you can use with Gitlab which will auto-create merge requests (MRs) every time you create a new branch on a project in Gitlab. (Optional) Git Feature Branches Feel free to skip this section if you are already familar with feature branch, skip to the Gitlab Auto MR section ...

Let’s take a look at how we can add offline local search 🔍 to a Gatsby blog. There are two main types of search we can use an offline search like elasticlunr and external API search engines like ElasticSearch. These are typically more scalable but also more expensive. You can find more info here. In this article, I will show you how to add offline search to your Gatsby blog using elasticlunr. This means your website needs to be indexed locally and will increase the bundle size as this index needs to be loaded by the client but with the scale and size of personal blogs (100s, not 1000s of blog post) this shouldn’t make a massive difference. We will also look at how we can add highlighting to our search results. ...

TailwindCSS allows us to use pre-defined classes instead of defining our CSS styles. In this article, we will go over how we can use Custom properties (sometimes referred to as CSS variables or cascading variables) with TailwindCSS. Setup First, follow the installation guide found here. This will show you how you can add TailwindCSS to your current project. For part 2 I will assume you called your CSS file global.css. This is the file that contains @tailwind base; etc. ...

This article (sort of) continues on from my previous article How to use Storybooks, Gatsby, Babel, Tailwind, Typescript together. In this article, we will document our React components using Storybook with MDX. You can find an example project using this here, you can also find a demo site for said project. Prerequisite Just to make sure everyone’s on the same page let’s follow the same steps to setup Storybook as we had in the last article. We will use the latest versions of Storybook (v6) so we can access the latest features. We will go over how we can use these features in the next article. ...

Recently I started to re-design my website, I decided to use this as an opportunity to learn some new technologies such as Gatsby, Tailwind. I also decided to try using Storybook. For this said project I used MDX to create my Storybook stories. In this article, I will show you how you can create Storybooks stories, for a Gatsby project with TailwindCSS, Typescript using MDX. You can find an example project using this here. You can also find a demo site for said project. ...

On a recent project, I was working on, I wanted to test my web service using docker-compose where I can run and kill Docker containers used by the application and see how my web application reacts to that. In this article, we will go over how you start docker containers using docker-compose from within Gitlab CI. The diagram above is a visualisation of what we are trying to achieve. We want to spawn Docker containers using docker-compose from within our job. The spawning and destruction of these Docker containers will be done via our Python code. We can achieve this by using dind (Docker in Docker). I have written a previous article on this topic which you can read more about here. This article assumes you already somewhat familiar with Docker, docker-compose and Pytest. ...

In this article, I will show you how you can test a Python web service that was built using Connexion (a wrapper library around Flask). We will go over how you can mock functions and how you can test your endpoints. There are two related articles I have written in the past listed below. In the first one we go over how to create a web service using Connexions, the same web service we will in this article. In the second article I introduce how you can use pytest-mock and pytest-flask to test a Flask web service. ...