To test the site locally, but on the simulator, I needed to modify the hosts file on the Android simulator to point the test domain to the host machine:

• Start the emulator in writeable mode:

•   emulator <avd-name> -writeable-system
  

• Set adb to root

•   adb root
  

• Remount filesystem

•   adb remount
  

• Log into the simulator shell

•   adb shell
  

• Edit the hosts file

•   vi /system/etc/hosts
  

After this, you should be able to access the FQDN on the simulator, which will now be pointing to the specified IP.

Recently, I got a new Windows machine from work and it started complaining that it couldn't use the right SSH key for my Enterprise GitHub repositories. No matter what I tried, it failed to authenticate correctly.

While, searching for a solution, I came across two ways to solve this:

Git config

Git provides a config parameter (git >v2.10) that can override the ssh command that git uses. So, piggybacking on this command, we can pass in the identity file information to it.

git config core.sshCommand "ssh -i ~/.ssh/id_rsa_github -F /dev/null"

What this enables is the ability to set a different SSH key for each repository as needed. Further, this method works across platforms.

Environment variable

Another method is to use the GIT_SSH_COMMAND (git >v2.30) environment variable to do the same. You can add an alias for git based on the site.

alias github='GIT_SSH_COMMAND="ssh -i ~/.ssh/id_rsa_github -F /dev/null" git'

And then use github clone repo_url instead of git clone repo_url.

I prefer using the git config method as I don't have to worry about which alias to use in which repository.

However, good engineers know that any feature can only ever be release-worthy, but never perfect.

The more a feature is delayed beyond what is necessary for it to work, the more disservice you are causing for people waiting on it.

Perfection is the enemy of utility.

Further, perfection cannot be achieved without feedback. Feedback after using a feature is worth 100x more than any analysis at the time of building it.

Although it is expected of the application to manage it’s own database migrations and seeding there maybe occasions where additional setup might be required such as enabling a database plugin, managing users and granting permissions. For such operations, using a Dockerfile, you would COPY an initialize shell script or SQL file to /docker-entrypoint-initdb.d. Any .sh or .sql file present there would be executed upon container start.

But while using docker-compose.yml to run an official database image, you can avoid creating a separate Dockerfile for the database by using the volumes parameter for the database service:

version: "3"
services:
  postgres:
    image: postgres
    environment:
      POSTGRES_USER: username
      POSTGRES_PASSWORD: password
      POSTGRES_DB: postgres
    volumes:
      - ./docker/init.sql:/docker-entrypoint-initdb.d/init.sql
    restart: always

And that’s it. Any SQL statements present in init.sql would be executed on container start.

P.S.: Using POSTGRES_USER POSTGRES_PASSWORD POSTGRES_DB environment variables would automatically create the database and the user and also grant permissions.

Note: Take care to append IF NOT EXISTS to any CREATE queries to avoid errors.

Install FontTools

Google has released an opensource font conversion program which we'll use. We'll need python, pip and some libraries - so, let's get them first.

Install python (v3)

Download and install python3:

• Windows (if you use WSL, see Linux installation, else: https://www.python.org/downloads/windows/)
• Mac (use homebrew - recommended, or get it here: https://www.python.org/downloads/mac-osx/)
• Linux (Debian-based - apt install python3)

Verify installation:

$ python3 --version
Python 3.8.5      # <-- expected output

Install pip (v3)

• Linux and Mac: $ python3 -m pip --version
• Windows: C:\> py -m pip --version

See: https://pip.pypa.io/en/stable/installing/

Verify installation:

$ pip --version
pip 20.0.2 from /usr/lib/python3/dist-packages/pip (python 3.8) # <-- output

Install FontTools and brotli

FontTools:

$ pip install fonttools
Collecting fonttools
  Downloading fonttools-4.21.1-py3-none-any.whl (849 kB)
     |████████████████████████████████| 849 kB 
Installing collected packages: fonttools
Successfully installed fonttools-4.21.1

brotli

$ pip install brotli
Collecting brotli
  Downloading Brotli-1.0.9-cp38-cp38-manylinux1_x86_64.whl (357 kB)
     |████████████████████████████████| 357 kB 
Installing collected packages: brotli
Successfully installed brotli-1.0.9

Download NotoSans

We test with NotoSans, a pretty heavy font, which makes the compressed sizes easy to compare.

Download the font from here: https://www.google.com/get/noto/ and decompress the contents.

We use the python3 REPL to try this out:

$ cd /path/NotoSans
$ python3
Python 3.8.5 (default, Jan 27 2021, 15:41:15)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from fontTools.ttLib import TTFont
>>> f = TTFont('NotoSans-Regular.ttf')
>>> f.flavor='woff2'
>>> f.save('NotoSans-Regular.woff2')
>>> f.flavor='woff'
>>> f.save('NotoSans-Regular.woff')
>>> exit()

Verify converted files:

ls -alh
  rwxr-xr-x    2   manoj   manoj      4 KiB   Sat Mar 13 13:35:12 2021    ./
  rwxr-xr-x   16   manoj   manoj      4 KiB   Sat Mar 13 14:17:12 2021    ../
  rw-r--r--    1   manoj   manoj    444 KiB   Wed Sep 20 17:20:30 2017    NotoSans-Regular.ttf
  rw-r--r--    1   manoj   manoj    233 KiB   Sat Mar 13 13:35:13 2021    NotoSans-Regular.woff
  rw-r--r--    1   manoj   manoj    163 KiB   Sat Mar 13 13:34:54 2021    NotoSans-Regular.woff2

As you see, woff2 provides the best compression for usage on the web, as long as you don't have to support IE. Check browser support here: https://caniuse.com/?search=woff2

To change the Dockerfile to perform trial and error means that you will have to rebuild it again - which is time consuming.

One way to keep a container running without having to do this, is to override the Dockerfile command from the docker-compose.yml file.

For example:

version: "3"
services:
  redis:
    image: redis
  web:
    image: web:v001
    build:
      context: .
      dockerfile: Dockerfile
    ports:
      - "3000:3000"
    volumes:
      - .:/opt/www
    depends_on:
      - redis

And in the Dockerfile you have a command:

FROM node:14-alpine

ENV APP_PATH /opt/www
RUN mkdir -p $APP_PATH

WORKDIR $APP_PATH

COPY package.json ./
COPY yarn.lock ./

RUN yarn

COPY . .

RUN yarn build

CMD yarn start

EXPOSE 3000

Anytime CMD yarn start fails, the container exits and is no more accessible for debugging.

Modify docker-compose.yml to include command override.

version: "3"
services:
  redis:
    image: redis
  web:
    image: web:v001
    build:
      context: .
      dockerfile: Dockerfile
    command: "tail -f /dev/null"     #   <--- <--- <---
    ports:
      - "3000:3000"
    volumes:
      - .:/opt/www
    depends_on:
      - redis

Now, when you start the container using docker-compose up -d, the container stays running.

Next, use docker ps -a to list all containers and obtain the hash for the container id.

Then, run sh within the container by: docker exec -it <sha> sh.

If everything goes well, you should see the shell prompt within the container:

/opt/www #

Now, you may debug for missing files, env vars, or even try to run yarn start and see why it is failing.

The Pi

The device is a Raspberry Pi 4B with 8GB RAM.

https://www.raspberrypi.org/products/raspberry-pi-4-model-b/

The case

owootec RetroFlag ResPi4. This case is an amazing combination of several things - the NES controllers to play Super Mario and Contra again (using RetroPie!). And also inbuilt interface for 2.5" SSD drive through USB 3.0. And a nifty case with a 32GB SD card and fan included. Highly recommend this one!

https://www.amazon.se/gp/product/B08FJ5BG52/

The SSD

Crucial CT240BX500SSD1. Although I was a little skeptical because not all SSDs are known to be compatible with the raspberry pi and I have had my share of trouble to get it to work!

https://www.amazon.se/gp/product/B07G3YNLJB/

The operating system

The operating system included in the SD Card that comes with the NesPi4 case included the Raspbian OS (default OS for the raspberry pi).

If the card is empty or contains a different OS, you may choose to install Raspbian on it. I used my ASUS laptop which had a SD card reader for this.

Raspi Imager Tool

This step is needed if the SD card doesn't already have Raspbian OS.

Get the raspi-imager from https://www.raspberrypi.org/software/ and install it.

Insert the SD Card and run the tool. The interface is pretty straightforward. Choose OS (in our case Raspbian - 32 or 64 bit does not matter), choose device, and click on Write. The tool downloads the prebuilt OS image and writes it onto the card. It also takes care of formatting and partitioning the disk correctly.

Configure

On first boot, the OS needs some setup. Choose the locale settings and connect to a network. Then choose a password and finish the rest of the setup wizard. Skip the reboot option.

Run the terminal application and:

sudo apt update -y

sudo apt full-upgrade

Once the update is complete:

sudo reboot

Verify

After reboot, verify that there are no errors and you are able to see the SSD drive correctly:

df -h

OR

lsblk

You should be able to see two disk drives - one for the SD card and the other for the SSD disk.

Setting up the SSD

We need to install the Ubuntu server image on the SSD and set it as the default boot option.

Install Raspi Imager

We need the Raspi Imager tool again. This time, the SSD drive acts as the target for the OS flash.

sudo apt install raspi-imager

Run it: Applications -> Accessories -> Imager

Choose the OS, choose the SSD drive as target and click Write.

For the OS, choose Ubuntu Server 20.10 - I faced multiple issues with the Ubuntu 20.04 LTS version of the image. Could be the combination of the SSD make/model or some driver files not working correctly (see note below).

Setup boot from USB

sudo raspi-config

Choose Advanced Options --> Boot order --> Select primary boot as USB, fallback to SD card option.

Accept and restart.

The device should now boot into Ubuntu Server 20.10.

User setup

The first start, the system prompts for login, use ubuntu for both login and password. It will complain that it is wrong, but it will auto-generate the password and present the prompt again.

Again, use ubuntu as login and password. This time, it will force to set a new password. Once the password is set, you will be able to login.

Configure Wifi

Since there is no UI, it is a bit tricky to setup the Wifi. Edit the netplan file:

sudo vi /etc/netplan/50-cloud-init.yaml

Add the wifis: section as below:

network:
  ethernets:
    eth0:
      dhcp4: true
      match:
        driver: bcmgenet .....
      optional: true
      set-name: eth0
  version: 2
  wifis:
    wlan0:
      dhcp4: true
      optional: true
      access-points:
        "SSID-NAME"
          password: "PASSWORD"

Save the file and apply changes:

sudo netplan apply

It will take some time for the network detection and connection to occur. Verify by:

ip a

Run the command several times until wlan0 status is UP and you see a local IP address assigned by DHCP.

Update system

Run:

sudo apt update -y

and:

sudo apt full-upgrade

Note: At the time of writing, the Ubuntu 20.04 LTS version failed to boot correctly multiple times, each time with a different issue -

• It failed to boot the first time, always defaulted to the SD card. When booting without the SD card, it failed to read the boot files correctly.
• On rewriting the OS again, it failed with "USB failed to respond to voltage check!" - which was possibly because of a power issue, but I knew it wasn't because the SSD had booted correctly with Raspbian OS before.
• Yet again, the SSD boot failed sometimes on the multicolor screen of death! Again attributed to power issues - I also tried a different power adapter, with no luck.
• I did get it to boot finally, but this time, it failed to recognize the WiFi network interface (service error on startup).

Finally, I decided to attribute it to the Ubuntu 20.04 OS image itself, and reluctantly tried the Ubuntu 20.10 version - which to my surprise worked like a breeze! No more issues.

While most entity changes (data type, column names, etc) are handled effectively by generated migrations, database seeding using TypeORM can be achieved using custom migrations.

In the official documentation for migrations, the examples show the use of raw queries and QueryRunner API to create migrations, I noticed that the QueryRunner instance also exposes the createQueryBuilder() method through the connection parameter. Therefore, you can use the same API that you would query the database from your application code to write the migration queries as well.

For instance,

export class seedTable1589221559012 implements MigrationInterface {

  public async up(queryRunner: QueryRunner): Promise<any> {
    await queryRunner.connection
      .createQueryBuilder()
      .insert()
      .into(<Entity>)
      .values([
        {
          id: 1,
          column1: 'abc',
          column2: 123
        },
        {
          id: 2,
          column1: 'def',
          column2: 456
        }
      ])
      .execute();
  }

  public async down(queryRunner: QueryRunner): Promise<any> {
    await queryRunner.connection
      .createQueryBuilder()
      .delete()
      .from(<Entity>)
      .where('id IN (0, 1)')
      .execute();
  }

}

This could help avoid having to write raw SQLs and enable type checking while inserting data.

In the software world - thanks to people who create and maintain open source libraries - there aren't many problems that aren't already solved.

In my quest to manage required environment variables for my Node.js applications, I stumbled upon envalid - a neat utility to ensure that all necessary variables are set before bootstrapping the application.

Manual validation

const cachePath = process.env.CACHE_PATH;
if(typeof cachePath === 'string') {
  // Perform other validations
} else {
  console.error('Missing CACHE_PATH environment variable.');
  process.exit(1);
}

// Repeat for more variables

With envalid

const {cleanEnv, str} = require('envalid');

// Verify required environment variables
const env = cleanEnv(process.env, {
  CACHE_PATH: str(),
});

Custom validators

Further, envalid supports custom validators to add additional checks.

const { makeValidator } = require('envalid');
const fs = require('fs');
const path = require('path');
const {cleanEnv, str} = require('envalid');

const exists = makeValidator(x => {
  // Correct path according to current platform
  const normalized = path.normalize(x);
  if (fs.existsSync(normalized)) {
    return normalized;
  } else { 
    throw new Error('Directory does not exist.');
  }
}, 'exists');

const env = cleanEnv(process.env, {
  CACHE_PATH: exists(),
});

console.log(env.CACHE_PATH);

Further

If you require custom error reporting, you can read more about it.