Blog

It’s no secret that GitLab is a beast of an application. As self-hosted git servers go, it’s easily the most powerful and feature complete. But that weight comes at a cost: resource usage. GitLab is no slouch, easily consuming upwards of 6GB of RAM by default without doing anything, and that’s even before you have a mountain of users trying to get their work done.…

Both LXC and Docker are great containerization technologies, brought to you by the powers of the Linux kernel. At their core, they’re pretty similar, but the further out you look, the differences increase massively. At their heart, they’re both still containers - understanding the differences between the 2 takes a bit of fundamental context:…

The internet is a pretty big place - a huge amount of data (approximately 131TB per second) is transferred over it every second of every day. The backbone of the internet is designed to transfer huge amounts of data, but people are impatient creatures, and want data as quick as possible.…

Docker containers (like onions) have layers. In your Dockerfile, each new RUN, COPY or ADD line creates a new layer (so do the others, but not ones which affect the filesystem). Each layer contains only the files which changed from the previous layer, which allows layers to be shared between containers, reducing download size and disk usage.…

Docker is a great containerization technology for running applications. It keeps multiple applications completely isolated from each other, only allowing connections exactly when you tell them to. But what if you’re on a hypervisor? You want your host OS to be as lean as possible (else it defeats the point), but you don’t want the overhead and additional complexities which come from full on VMs?…

For anyone who’s spoken to me, they’ll know I’m very quick to link people to posts I’ve written. That’s not in terms of pushing the things I’ve written (usually), but also being able to retrieve the links as quick as possible.…

I have a bunch of VMs and LXCs on my proxmox server. Whilst I like to keep as little data in each of them as possible, and instead mount in my storage (in the form of a ZFS and snapraid pool), I still use proxmox’s built-in backup feature to back up to a separate drive, because backups can’t hurt.…

My website is built with Hugo, a great static site generator with a bunch of features. One of the big missing features though is search. Hugo has documentation on a bunch of different integrations, but none of them quite did what I wanted.…

Static sites, ie those which are just files on disk rather than requiring a custom application or database to run, are incredibly simple to write. You can either do it yourself from scratch with a bunch of HTML, CSS and JS files, or use a generator like Hugo or Zola.…

As a developer I do basically everything in git and for fun I run my own git server on my home server. I’ve swapped around quite a lot between GitLab and Gitea, but finally settled on GitLab. It’s a bit heavy, but the deep CI integration is really nice.…

Backups are critical to any systems longevity and reliability. If you’re not backing up your data, stop reading this now, go do it, then come back… Assuming none of you suddenly panicked and left, let’s keep going. You can keep telling yourself otherwise, but eventually, every system will experience some kind of failure.…

My website is a very important project to me. I’ve written a lot of content over the years, useful both to me and other internet folks. Currently, my website is a static site, powered by Hugo. Because it’s static, the content is served insanely quickly and handles any insane load spikes like a champ (not that any have happened).…

Back in December, I fully rebuilt my home server from the ground up based on Proxmox. Being a hypervisor OS, it makes sense to run everything in VMs or LXC containers, not on the host. Therefore, there’s a huge amount of scope for opinions, lessons and customizations.…

YouTube is the largest video hosting platform on the planet, serving well over 1 billion videos a day to its over 30 million daily active users. People have had issues with YouTube’s subscription service for years, and for a service so large and so widely used it seems crazy that one of its fundamental features is so complex and illogical.…

It’s that time again, time for another Django LTS release. Since Django 2.2, back in 2019, a lot has changed in tech, in Python and of course in Django. Historically, I’ve worked entirely on LTS versions, hence combining these 3 releases together.…

ZFS is a great filesystem, especially for any kind of data storage, but the fact it’s not integrated into the Linux kernel makes it a risky choice for the root OS. Canonical are making this easier for Ubuntu users by tightly controlling and testing the kernel and ZFS to ensure the versions integrate correctly, reducing the risk of the system failing to boot - but the odds aren’t 0.…

At some point, servers need to be put on the public internet. Whether that be a VPS in the cloud, or your new homelab. Once a server is on the internet, it’s subject to anything and everything the internet has to offer, from botnets to hackers and script kiddies.…

Nebula is a great mesh network I recently deployed into my stack. For connecting nodes spread between networks, it’s great, much better than my previous WireGuard installation. An additional feature of nebula is unsafe_routes. Unsafe routes allow nodes which don’t have Nebula installed to be accessible to other Nebula nodes.…

Yesterday, I moved my Git server from GitLab to Gitea. There’s nothing wrong with GitLab, I actually quite like it, but it’s a rather large tool for my needs. Gitea is much more lightweight, faster, and provides all its features for free.…

WireGuard has been the “hot new thing” when it comes to VPNs, but it’s not always the best suited for every workload. Nebula is a mesh network originally created by Slack, but now owned by a separate company. TechSNAP 419 - Nebulous Networking Linux Unplugged 329 - Flat Network Truthers # What’s a mesh network?…

It’s that time of the year again: time to look back at how I work, the tools I use, and how the next year might look. I’ve been working from home basically full time since the UK went into lockdown 17th March.…

GPU Passthrough has become a great way to run a Linux host, but still run games under Windows. By having 2 GPUs, 1 for the Linux host and the other for a Windows VM, you give Windows its own full GPU for games, but without having to run Windows as your primary OS.…

NFS is a great protocol for sharing files quickly and simply over the network. Whilst it’s not designed for end user use, it’s great for mounting directories from remote machines, and having them be performant. NFS' lack of authentication is in a way a feature, honest.…

You should back up your data, properly! If you’re not, you’re playing a dangerous game with fate. Computers are pretty reliable, but they also go wrong, often. You should always backup your files, but backing up a containerized application isn’t quite as simple.…

Black Friday, the only day which seems to last over a month, is a great time to buy tech. Whilst most people will be buying early christmas presents, new gadgets for themselves, or just impulse buying stuff they don’t need (something I totally never do…), I ordered the parts for my new server.…

People say there’s no 100% reliable way to wipe a storage drive, and they’re right. By the nature of how mechanical drives work, there’s no real way to say for sure whether the data is ever really gone. With drives, the only way to be sure the content is gone is to destroy them.…

I recently added some RGB LED strips around my headboard and bed frame, because everyone needs more RGB in their life. The only thing better than RGB is internet connected RGB. One of the most common controllers for this is the MagicHome.…

This year’s Hacktoberfest was a little different, mostly in quite how much the internet gave a fuss about it. Given quite how much of the internet was talking about it back in early October, it doesn’t take long to find information and context on why it was so controversial.…

Recently, I started setting up a new application on my docker host. It was late in the day, and I just wanted to get something up and working to play around with. Just my luck, I was met with wonderfully cryptic error:…

I recently had a far from fun morning. I had my website cloned on my desktop, with some commits which weren’t upstreamed. Having done some work on my laptop, upstream was ahead of my desktop, which needed rectifying. git pull -r is a great feature of git, which does a rebase rather than a merge commit when pulling a diverged remote.…

The internet is a wild place, filled with well, everything. There are many ways of exposing an application to the internet, but no matter how secure an application claims to be, or how confident you are with your infrastructure, sometimes you may just be more comfortable keeping it internal.…

After recently deploying a ZFS pool, I realized I had little insight into the health of my drives. I can run SMART stats now and then, but that’s not quite the same. # Scrutiny Scrutiny is a tool to help you with just that.…

I’ve been using Traefik for a while now, and I’ve helped quite a few people with it, but I still see a lot of people scared off it. This isn’t helped by a lot of guides being incredibly verbose, and not explaining what’s going on very well.…

Traefik’s dashboard is a great tool to diagnose routing issues, and check services are being detected correctly, but it can’t do much more than that. It doesn’t show any metrics. Instead, it relies (arguably correctly) on external monitoring tools for metrics.…

ZFS is a pretty sweet filesystem, an opinion shared by the majority of the internet it seems. BTRFS (No Wikipedia, it’s not “butter fuss”) is also a nice filesystem, and the fact it’s built-in to the kernel makes it far safer to use as a root filesystem.…

Clickhouse is a column oriented database engine, mostly designed for analytics. It was developed and deployed by Yandex in their analytics offering, but Plausible, my analytics provider of choice, uses it to store the analytics data, as it’s more suited to this than PostgreSQL.…

Recently, I bought myself a new server from Kimsufi, to function as an off-site backup server. And after fighting with both their management interface and customer services, I finally got it booted. Kimsufi have a respectable list of OS options, however not especially recent.…

Docker containers, and containers as a whole, are really just a regular program wrapped in some extra protections provided by the kernel (namely cgroups etc) to create isolation, and other interesting features. Unlike VMs, containers run closer to the host operating system, so close they use the same kernel, meaning it’s even more important to protect it.…

The majority of England (where I live) is being hit by quite a heatwave. It’s hot here, like really hot! Temperature recordings from my bedroom this past few days Us brits are incapable of handling the heat, and enjoy nothing more than telling each other how hot and unbearable it is.…

I only recently talked about using GoAccess as an analytics tool. Over the last couple months, I’ve been more interested in how many hits this website actually gets. GoAccess was a really simple solution which worked on log files and only captured the bits of information I really cared about: Page views.…

Last year, I switched all of my hosting from arbitrarily installed packages to Docker. This made installing and configuring incredibly simple, but updating a little less defined. Whilst Docker itself is updated through the system package manager (probably), the containers themselves aren’t.…

Django already does some pretty incredible things when it comes to performance, especially in the ORM layer. The lazy loading, fluent interface for querying means it’ll only fetch the data you need, when you need it. But it can’t handle everything for you, and often needs some help to work out what you’re doing, and what exactly it can optimize.…

I write quite a lot. Whether it be coursework, blog posts, or any other random thing. Most of my time is spent in either VSCode or QOwnNotes, neither of which have any kind of spelling and grammar checking. It’s rare I’ll actually go back over something and fully check its grammar, unless there’s a very obvious issue or typo.…

Storing passwords is a pretty simple problem in software development, right? Wrong! Storing passwords correctly is pretty complicated. With that said, it’s very simple to just lean on work someone else has done, and the libraries available for your language of choice.…

Duplicati is one of my favourite backup system. It’s pretty fast, supports numerous backup sources, and has a nice configuration web UI. Unfortunately however, it can’t be used to back up remote files. In fact, I can’t find a nice fully-features backup solution which does do this, which sucks.…

Traefik is a cloud native reverse proxy, which is basically a fancy way of saying it’s a reverse proxy with some fancy features. Specifically it has fancy features around auto-discovery, and deep integration with technologies like Docker and Kubernetes. # Basic concepts Traefik has four fundamental concepts: Entrypoints, routers, middleware and services.…

In the current lockdown situation, a lot of people are starting to eye up that old desktop machine, or Raspberry Pi they bought for a project and just left on a shelf, and thinking of putting it to use, as a server!…

As a part of Nextcloud 17, Nextcloud released Text, a fancy new WYSIWYG editor for markdown with support for collaboration. Personally, I quite liked the old interface: A simple plaintext editor with syntax highlighting, paired with a side-by-side preview. Nextcloud Text was a step in the wrong direction for me, but I completely understand why Nextcloud did it.…

A few days ago, I was sharing a blog post to someone on the self-hosted podcast discord, and they asked if I was self-hosting my website. Unfortunately, and rather ironically, I had to answer no. I’ve been intending to move it over to my own server for a while, so this felt like as good of a push as any!…

Recently, I decided to put some analytics on my website. Would be nice to see what view number are like and what pages get the most traffic. Most people would just stick Google Analytics on and be done with. But the privacy implications off that are huge and terrible, not to mention any self-respecting privacy extension would block it almost immediately.…

Last year, I wrote a post on setting up a gateway to a private network, powered by OpenVPN-AS. I ran this network setup for quite a while with a lot of success, exposing services on my home network to the public internet, securely.…

# What is WireGuard? The website defines it as “… extremely simple yet fast and modern VPN that utilizes state-of-the-art cryptography.”. Which basically means it’s a VPN, but sane. The point of a VPN is to allow two machines to talk to eachother, no matter how the network inbetween is set up.…

When disabling a user account on a Linux box, it is good practice to also change the shell to something which, well, isn’t a shell. The point of these shells is rather than presenting the user with a prompt to execute further commands, it returns a failure code, and log out the user.…

This year marks the 3rd year of my pattern for publishing a “My Stack” post, noting down how I get work done both professionally and personally (2018 / 2017). This year, I’m stealing borrowing inspiration from CGP Grey / Cortex and their “State of the Apps” episodes, and also talking about productivity, and the other services I use to get things done.…

Yesterday, an email was sent to django-announce, informing of an upcoming security update, labelled “high” severity. Previous notifications like this have been one week before the actual disclosure; This email, just 12 hours. The updates were scheduled to be released 12:00 UTC the next day (today).…

This is year number three of my participation in Hacktoberfest, the initiative from DigitalOcean, and new this year, Dev.to. In previous years, the objective was to submit five pull requests to an open-source project. This year, the number was reduced to four, for some reason.…

For the last four years, I’ve been spending every working day off a Dell Optiplex. With an after-market SSD upgrade, and a little extra memory, it makes a pretty good work machine. When it comes to needing to work away from my desk, it’s a little annoying having only a desktop.…

WireGuard is taking the VPN world by storm, coming very close to the current champion OpenVPN in simple, small-scale deployments. It’s just unfortunate few people know about it, and quite how incredible it is! # What is WireGuard? WireGuard® is an extremely simple yet fast and modern VPN that utilizes state-of-the-art cryptography.…

I’ve been an arch user for many years, and a linux user for even longer, but I’ve never installed arch from scratch. I was an Antergos user for many years, but after its demise, I needed an alternative. In a previous post, I spoke of attempting to install vanilla arch from scratch on my laptop.…

I’ve been an Antergos user for almost three years, and I love it! It’s like Arch, but with a simple installation process, and yields a near-pure Arch install, unlike Arch derivatives like Manjaro. Unfortunately, on 21st May 2019, the Antergos project ended.…

Spoiling films is something which has plagued people since forever. If something dramatic happens, of course people don’t want to find out by just being told, they want to watch and experience it for themselves. Unfortunately, it seems many people take pride and pleasure in spoiling films for people.…

Object-Oriented Programming (OOP) teaches that classes can have two kinds of attributes: Instance and Static. Instance variables are attached to a specific instance of the class, and each has separate memory locations. Static variables are tied to the class itself, and are shared between instances.…

April marks the release of Django 2.2, the latest LTS version of the popular Python web framework. Django 2.2 marks almost two years of development since the last LTS release, 1.11 in April 2017, and brings with it some very large improvements and changes which naturally come with a major version bump.…

# Cleaning up Facebook - Part 1 Around 10 years ago, Facebook was everything. No matter your interests, you could find like-minded people on Facebook to share your passion with. But that communication came at a price: your personal data. In recent years, we’ve realized to what extent Facebook was selling our data and generally invading our privacy.…

Last year, I wrote My stack 2017, and now it’s time for a follow-up. A year is a long time, and my day-to-day stack has changed quite a bit since then. # OS I’m still an Antergos user, and have no sign of changing.…

Hacktoberfest is a great initiative created by DigitalOcean and GitHub to get more developers contributing to open-source projects. After opening a set number of pull requests to open-source projects, they’re rewarded with branded swag. As someone who does a lot of development in their free time, it’s not hard for me to find some open source projects, and contribute where possible.…

In terms of security, WordPress, and PHP in general for that matter, have become a bit of a joke. If you want a site to be secure, people tend to steer clear of WordPress and PHP. That being said, nothing stands even close to WordPress in plugin support, community size, and documentation.…

VPNs are a way of accessing application which sit on a separate network using an encrypted tunnel. Contrary to popular belief, they are not designed to anonymize your internet habits. Whilst VPNs are designed to enable a client to access the servers network, it’s possible to use them to provide a server to access to its client’s network.…

I’ve been using KeePassXC since not long after it’s initial split from KeePassX in late 2016. I’ve bounced around many password managers, but KeePassXC looked to fill all the boxes: It’s actively maintained, unfortunately unlike KeePassX It’s open source, and easy to contribute to, as I have It’s got all the features I need, like TOTP and Browser Integration Cross platform (not KeePassXC specific) 2.…

Bluetooth audio is great! I’ve had a pair of Bluetooth headphones, Sony MDR-XB950 BT, for around a year now, and the ability to listen to music without cables is amazing. But, I can only use the Bluetooth parts of this with my phone, because on Linux, it just sounds terrible.…

My university has a development sever, which it uses to host our coursework without the need to set up a development environment locally. It also enables lecturers to mark our work in a controlled environment, without needing to spin up an environment, and run untrusted code on their machines, a security hole I’m more than likely to take advantage of!…

As a software engineer and perfectionist, I have my machines set up in a very specific way, so I can do my job properly and have everything just the way I like it. Thanks to my dotfiles, I have everything syncing up between machines, meaning the tools I use are configured correctly, the same, everywhere.…

As anyone who’s used an application written with the QT UI framework will know, they don’t always look the best, and certainly don’t fit in with the rest of your desktops theme in the way GTK does. Certain themes support styling both GTK and QT applications, however most don’t.…

I’ve had a website for around four years now, starting with a python CGI-based site hosted at 1&1, and evolving into its current form, powered by Hugo. Although I’m a web developer, I’m very far from a designer. I really can’t design anything!…

I’m one of those Arch users who apparently doesn’t use arch properly: I install updates daily, including packages from the AUR. This has the great benefit of giving me the most up-to-date packages available from upstream. However, the downside of meaning I have the latest packages from upstream, meaning if something breaks, even temporarily, it breaks for me.…

As it is Cyber Security Month, now is the perfect time to work on improving the security on my websites, projects, and servers. But, upgrading them for now isn’t good enough for me, I wanted to add a way of scanning projects automatically, to check for any new issues.…

Running steam on linux is great, besides the triple-a game support obviously. But running it on an iGPU has recently been causing me problems, mainly under Arch. # Stacktrace: jake@***:~$ steam ~/.local/share/Steam/steam.sh: line 154: VERSION_ID: unbound variable ~/.local/share/Steam/steam.sh: line 154: VERSION_ID: unbound variable Running Steam on antergos 64-bit ~/.…

Throughout my life, I’ve had numerous email providers, starting with Hotmail almost 10 years ago. Recently, I’ve been focusing more on ways I can secure my emails. No, I may not have anything to hide, but that doesn’t matter! Originally I thought the best way to keep things secure, and out of the hands of any government body was to host it all myself.…

Recently, at DabApps, we’ve been migrating our mobile app workflow over to using React Native rather than Ionic, mainly because of its near native performance. For the first few projects, there were only a couple of us that knew how to use React Native effectively, and work around the qwerks it has.…