Saturday, October 9, 2021

periodic renewal of https letsencrypt certificates

Confirm expiration date of current certificates
openssl x509 -dates -noout < \
/home/pdg/proofofconcept/v7_pickle_web_interface/certs/fullchain.pem

Delete existing certs
sudo rm -rf /etc/letsencrypt/live/derivationmap.net
sudo rm -rf /etc/letsencrypt/renewal/derivationmap.net
sudo rm -rf /etc/letsencrypt/archive/derivationmap.net
Confirm folders are empty
sudo ls -hal /etc/letsencrypt/live/
sudo ls -hal /etc/letsencrypt/renewal/
sudo ls -hal /etc/letsencrypt/archive/

Request new certs
sudo certbot certonly --webroot \
-w /home/pdg/proofofconcept/v7_pickle_web_interface/certs \
--server https://acme-v02.api.letsencrypt.org/directory \
-d derivationmap.net -d www.derivationmap.net
Output should be something like
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator webroot, Installer None
Obtaining a new certificate

IMPORTANT NOTES:
 - Congratulations! Your certificate and chain have been saved at:
   /etc/letsencrypt/live/derivationmap.net/fullchain.pem
   Your key file has been saved at:
   /etc/letsencrypt/live/derivationmap.net/privkey.pem
   Your cert will expire on YYYY-MM-DD. To obtain a new or tweaked
   version of this certificate in the future, simply run certbot
   again. To non-interactively renew *all* of your certificates, run
   "certbot renew"

Copy new certs to directory that nginx mounts in Docker-compose
cd /home/pdg/proofofconcept/v7_pickle_web_interface/certs

mv dhparam.pem dhparam.pem_OLD
mv fullchain.pem fullchain.pem_OLD
mv privkey.pem privkey.pem_OLD

sudo cp /etc/letsencrypt/live/derivationmap.net/fullchain.pem .
sudo cp /etc/letsencrypt/live/derivationmap.net/privkey.pem .
sudo chown pdg:pdg privkey.pem
openssl dhparam -out dhparam.pem 2048

Restart Docker-compose
cd /home/pdg/proofofconcept/v7_pickle_web_interface
docker-compose up --build --force-recreate --remove-orphans --detach
If the docker containers are not restarted, the changes made to the file on the host won't take effect.

Verify in a browser that https://derivationmap.net/ has the updated certificate. 

Set a calendar reminder to renew the certificate.

Saturday, August 21, 2021

documenting my frequent pivots

  • survey all topics in Physics. [See this 2017-07 post.]
  • investigate refactoring the database from JSON to RDF. [See this 2020-12 post.]
  • goal: add a new derivation to the Physics Derivation Graph to identify bugs and inefficiencies in the workflow
  • instead of entering a new derivation, new goal is to create tutorials for an existing derivation. [See this 2021-08 post.]
  • before creating tutorials, write a "gold standard" for one derivation (then create tutorials). [See this 2021-07 post and this page.]
  • instead of fixating on a "gold standard," refocus on the original intent of the project: connecting expressions from the breadth of Physics. [See this 2021-08 post and this page.]

Friday, August 20, 2021

refocusing the motivation for the Physics Derivation Graph

The following aspects are "nice-to-have" in the Physics Derivation Graph:

  • validation of derivation steps using computer algebra systems
  • validation of consistency expression dimensions
  • validation of consistency of units when present in expressions
  • visualization of the graph
  • make the code pretty and easy to navigate for contributors
  • document the source code and design decisions
  • searching the graph (see this and this and this)
  • ability to determine whether paper is written by a crackpot
  • ability to detect unintentional errors in an article or book
  • cross-document references
  • semantic tagging of Latex documents
  • converting Latex from arxiv into formats supporting the above objectives
  • linking Physics Derivation Graph to existing ontology databases

The core objective of the Physics Derivation Graph is identifying the mathematical connectivity of the various domains of Physics. The same connectivity should be able to relate "basic" Physics (e.g., F=ma) to advanced Physics (e.g., the theory of the Standard model, string theory). 

I know of two ways to document the connectivity:
  • identify symbol re-use
  • identify inter-related derivations at the level of expressions
The symbol re-use is less interesting than the re-use of expressions. 


I've previously documented a plan,
but I didn't specify what would qualify as sufficient to show completeness. 

What core expressions are representative of all of Physics? Are there a set of expressions that, if shown to be connected by derivations, would be sufficient to demonstrate the span of the concept?

I expect that, for any given subdomain of Physics, there are central expressions. The task list is then
  1. identify every named expression in Physics -- see https://derivationmap.net/central_expessions
  2. identify every variable in that list of named expressions
  3. enter all named expressions and variables into the PDG database
  4. determine which expressions are not connected to anything

searching the Physics Derivation Graph database

The Physics Derivation Graph database is just a JSON file, so searching a plain text file is easy. However, the value of search is in the relations, not just finding a match. 

There are multiple aspects of the Physics Derivation Graph database to search: symbols, units, dimensions, inference rules, symbol names, operators. 

A workflow for search is

  1. user searches web interface for "x"
  2. the search interface returns multiple rows of a table; something like
    symbol IDsymbolnamedimension units
    948294 x_m asdfL:0,T:1,M:0
    113942 y_x mimiL:-1,T:0,M:0Kg
    501901 XM minXL:0,T:2,M:1 seconds^2
  3. if the user selects one of the symbols, then
  4. the search interface shows which expressions contain that symbol
  5. if the user selects one of the expressions, then
  6. the search interface shows which derivations contain that expression

Currently, the search of symbols ( https://derivationmap.net/list_all_symbols ) is separate from the search of expressions ( https://derivationmap.net/list_all_expressions ) though the two are hyperlinked in both directions.

Sunday, August 8, 2021

troubleshooting why docker-compose does not successfully launch locally

Normally I make changes to the repo https://github.com/allofphysicsgraph/proofofconcept/tree/gh-pages/v7_pickle_web_interface and then run git pull on my DigitalOcean VPS Droplet. It has been a long time since I tried running the web server locally on my laptop. 

Here's the process I went through to get the web server running locally.


First I had populated the "certs" directory

/Users/username/version_controlled/allofphysicsgraph/proofofconcept/v7_pickle_web_interface/certs
on my laptop from the remote VPS. 


Then I had to create 

/Users/username/version_controlled/allofphysicsgraph/proofofconcept/v7_pickle_web_interface/.env
with the Google variables for login authentication. 


In the directory on my laptop

/Users/username/version_controlled/allofphysicsgraph/proofofconcept/v7_pickle_web_interface
running the command docker-compose up --build failed. Specifically, the nginx and flask containers would start, but nginx would fail because the flask container wasn't responding. I added
restart: on-failure
to the nginx section in docker-compose.yaml

That didn't solve the flask issue, but it allowed the containers to persist while I inspected the logs. Even though I couldn't enter the flask container, I could review the logs produced by gunicorn by running the command

tail -f flask/logs/gunicorn_error.log
which showed the causal issue
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/appuser/app/wsgi.py", line 15, in <module>
    from controller import app
  File "/home/appuser/app/controller.py", line 63, in <module>
    from secure import SecureHeaders  # type: ignore
ImportError: cannot import name 'SecureHeaders'
[2021-08-08 19:56:37 +0000] [11] [INFO] Worker exiting (pid: 11)
[2021-08-08 19:56:37 +0000] [1] [WARNING] Worker with pid 11 was terminated due to signal 15
[2021-08-08 19:56:37 +0000] [1] [INFO] Shutting down: Master
[2021-08-08 19:56:37 +0000] [1] [INFO] Reason: Worker failed to boot.
Root case: In my requirements.txt I hadn't pinned the version of the Python library secure. According to this issue there was a recent update . I ended up pinning secure==0.2.1 in requirements.txt

Now I am able to run docker-compose up and get a web page at https://localhost/

domains in Physics that are difficult for the Physics Derivation Graph

Geometry-based derivations

The spatial reasoning needed does not conform to a computer algebra system


Experiments

The chain of reasoning needed to justify design choices is difficult. The motivating insight is subjective.

consolidating content from Physics Derivation Graph websites

This post outlines the current resources associated with the project, and serves as a reminder to the author for what actions need to be taken. 

https://derivationmap.net/ -- primary website with content

https://github.com/allofphysicsgraph/proofofconcept -- website for source code

https://physicsderivationgraph.blogspot.com/ -- blog containing ideas and design decisions


https://allofphysicsgraph.github.io/ redirects to https://derivationmap.net/


TODO

https://github.com/allofphysicsgraph/proofofconcept/wiki -- unorganized wiki; should be consolidated with https://derivationmap.net/


https://sites.google.com/site/physicsderivationgraph/home -- old website, not maintained; should be consolidated with this blog.