I’ve been trialing GitHub Copilot for a couple of weeks now. I can say that it’s well worth it especially when writing code from a blank slate. It outputs code blocks that help my brain process more quickly.

Its output rarely works as-is, but it only needs minimal tweaking.

It’s another subscription though, and I try to limit my monthly expenses to a minimum. It’s a nice-to-have, but not a necessary-to-have with the amount of coding I do at the moment.

When I was migrating my old server, I saw in the Vultr Marketplace FauxPilot. It turned out it’s an open-source alternative to GitHub Copilot server using Salesforce CodeGen.

I just had to try it.

Requirements

The only physical requirement is an Nvidia GPU with CUDA support (I have a Tesla P4 ✅).

The rest are just software to install:

• Docker
• Nvidia Container Toolkit (Install Guide)
• curl and zstd

Server Installation

I use Ubuntu 20.04 as my base OS. From a fresh install, here’s what I needed to do to run FauxPilot.

1. Install Docker

curl https://get.docker.com | sh \
  && sudo systemctl --now enable docker

2. Install Nvidia Container Toolkit

distribution=$(. /etc/os-release;echo $ID$VERSION_ID) \
  && curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey \
  | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
  && curl -s -L https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.list \
  | sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' \
  | sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

sudo apt-get update
sudo apt-get install -y nvidia-container-toolkit
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

3. Install Nvidia CUDA drivers

wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb
sudo apt-get update
sudo apt-get -y install --no-install-recommends cuda
sudo apt-get -y install nvidia-driver-535

Make sure the drivers are properly installed by running nvidia-smi.

4. Clone FauxPilot repo

git clone https://github.com/fauxpilot/fauxpilot.git
cd fauxpilot

5. Run FauxPilot setup

sudo bash ./setup.sh

I chose codegen-2B-multi because my GPU only has 8GB VRAM, and I’m coding in PHP. Higher parameter models require more VRAM and RAM.

6. Launch FauxPilot

sudo bash ./launch.sh

Demo video: /media/homelab/2023/08/Screen-Recording-2023-08-24-at-22.00.05.mov

Test if the server is working:

curl -s -H "Accept: application/json" \
  -H "Content-type: application/json" \
  -X POST \
  -d '{"prompt":"def hello","max_tokens":100,"temperature":0.1,"stop":["\n\n"]}' \
  http://localhost:5000/v1/engines/codegen/completions | jq

Response should be:

{
  "id": "cmpl-OCButmOAbNedOMOxjPc0v9skuLdk7",
  "model": "codegen",
  "object": "text_completion",
  "created": 1692885668,
  "choices": [
    {
      "text": "(self):\n return \"Hello World!\"",
      "index": 0,
      "finish_reason": "stop",
      "logprobs": null
    }
  ],
  "usage": {
    "completion_tokens": 11,
    "prompt_tokens": 2,
    "total_tokens": 13
  }
}

Client Setup

Now that I have a working server, I need to setup my client. There’s a VSCode extension called FauxPilot. The only configuration change needed is pointing to the server address. After that, it works right away.

Demo video:

The suggestions quality is very far from GitHub Copilot. But at least it works!

There are a lot of factors why it’s underperforming. Maybe it’s the model itself, or the size of the model I chose, limited context, difference in training data.

Regardless of the quality, it’s exciting that it can be ran locally with old techs.

I’ve been wanting a solution to have conversation

Wow. This app mindblowingingly easy to install and use.

This is for managing and consuming audiobooks. I actually gave up trying to manage my own audiobooks and subscribed to Audible instead. The issue is I only have phases on when I listen to audiobooks. It only recently started when I started driving everyday again to pickup my kids from school.

The issue I have with Audible is it’s a subscription. And I don’t really own books I purchased there. It’s only rented until they close down.

With that concern, I’ve attempted to download copies of my books. But managing it as a flat file is hard. I have a folder “Audiobooks” where my audiobooks are left to be ignored.

Ang hirap kasi i-navigate. Opening it with the Music app, it will import the whole thing. Without context too, walang chapters, walang metadata.

This app solves all those. Grabe. The interface is super delightful to use to. As soon as I upload 1 book, I started listening. That’s it.

Wow.

Bigla kong na-miss yung Google Reader. I used to subscribe to blogs. Madami pa ring blogs ngayon but it’s hard to keep track unless may aggregator.

I started looking for self-hosted solution na may similar user experience with Google Reader. Yung naka-open na yung articles but scroll to read lang. FreshRSS fits the bill.

I’ve been meaning to set it up kasi may Docker package naman. I thought it will only take less than 30 minutes pero it’s 1.5 hours bago ko napaganda. It’s mix of where my containers are hosted, yung provided nilang default config na hindi gumagana out-of-the-box, and small changes sa config na I would not thought will break my initial installation.

But now it works:

Here’s to more curated way to consume content from the web!