Sunday, October 11, 2020

upgrading Ubuntu 18.04 to 20.04 on DigitalOcean VPS droplet

I've been running a DigitalOcean droplet for $5/month for the past 6 months. Because I was new and didn't know better, I selected the Ubuntu 18.04 droplet. 

Now I want to update to Ubuntu 20.04 LTS. 

The guide recommends starting with a fresh 20.04 image instead of upgrading. 

The following is a record of the steps I took in this process. 

Total duration: 2 hours. The process took longer than expected because I hadn't previously configured the website from a bare Ubuntu server. Also, I had made a few changes since the initial installation that weren't documented.

Step 1: collect all data prior to turning off the server

Used scp to copy data from the droplet to my mac

scp user@IP:/home/pdg/arxiv_rss/rss_filter_email.py .
scp user@IP:/home/pdg/arxiv_rss/.env .
scp user@IP:/home/pdg/videos/* .
scp user@IP:/home/pdg/.bash_history .
scp user@IP:/home/pdg/.bashrc .
scp user@IP:/home/pdg/.python_history .
scp user@IP:/home/pdg/.sqlite_history .
cd proofofconcept/v7_pickle_web_interface/
scp user@IP:/home/pdg/proofofconcept/v7_pickle_web_interface/.env .
scp user@IP:/home/pdg/proofofconcept/v7_pickle_web_interface/certs/* .
scp user@IP:/home/pdg/proofofconcept/v7_pickle_web_interface/flask/logs/* .
scp user@IP:/home/pdg/.ssh/authorized_keys .

Grab the crontab entry

0 0 * * * /usr/bin/python3 /home/user/arxiv_rss/rss_filter_email.py >> /home/user/arxiv_rss/cron.log 2>&1

Step 2: power off the server and take a snapshot

https://www.digitalocean.com/docs/images/snapshots/how-to/snapshot-droplets/

Step 3: Start a new droplet

Selected Ubuntu 20.04

Step 4: configure accounts and access

adduser pdg
usermod -aG sudo pdg

ufw allow OpenSSH
ufw enable

Instead of creating new SSH key pairs, 
I imported my authorized_keys file to /home/pdg/.ssh/

To get the authorized_keys file I temporarily allowed password-based authentication for scp using
sudo vim /etc/ssh/sshd_config
change "PasswordAuthentication No" to "PasswordAuthentication Yes"
sudo service ssh restart
While I was there, I also ran
change "PermitRootLogin yes" to "permitRootLogin no"
Once I had transferred the authorized_keys file, I reverted to "PasswordAuthentication No" and ran
sudo service ssh restart


sudo ufw allow 443
sudo ufw allow 80

Step 5: update OS


sudo apt-get update
sudo apt-get upgrade

Step 6: install metrics


sudo apt-get purge do-agent
curl -sSL https://repos.insights.digitalocean.com/install.sh -o /tmp/install.sh
sudo bash /tmp/install.sh
/opt/digitalocean/bin/do-agent --version

Step 7: install Docker and Docker-Compose


Step 8: certs

sudo apt install certbot python3-certbot-nginx
sudo certbot certonly --webroot \
     -w /home/pdg/proofofconcept/v7_pickle_web_interface/certs \
     --server https://acme-v02.api.letsencrypt.org/directory \
     -d derivationmap.net -d www.derivationmap.net

Your certificate and chain have been saved at:
   /etc/letsencrypt/live/derivationmap.net/fullchain.pem   Your key file has been saved at:   /etc/letsencrypt/live/derivationmap.net/privkey.pem   Your cert will expire on 2021-01-09.
https://security.stackexchange.com/questions/94390/whats-the-purpose-of-dh-parameters
cd /etc/ssl/certs
sudo openssl dhparam -out dhparam.pem 4096
cp dhparam.pem ~/proofofconcept/v7_pickle_web_interface/certs/

Step 9: restore data from backup

git clone https://github.com/allofphysicsgraph/proofofconcept.git
scp .env user@IP:/home/pdg/proofofconcept/v7_pickle_web_interface/
cd proofofconcept/v7_pickle_web_interface/flask
cp users_sqlite.db_TEMPLATE users_sqlite.db
cd ..
docker-compose up --build --remove-orphans --detach