This post defines discrete levels of software quality. There are many dimensions of what "software quality" means, so the discrete levels described here are an oversimplification. The attribute clusters below are typically observed together.
software quality level 1
"Worked for me at one point."
Code does not compile
No documentation
Required dependencies not included
Purpose of code is unclear
Hard coded numerical values
Code is wrong - severe design and implementation flaws
BLUF: Definitions of distance and modularity enable characterization of complexity in technical and social systems. However, I am unable to find a simple quantification of complexity or rank complexity.
This post is a mixture of notes from a YouTube video (linked at the bottom of this post) plus some analysis.
Characterizing Complexity
The scope of this post covers the both software complexity and social complexity.
Software complexity is characterized by
how many layers of abstraction: namespace/package, classes, functions?
the number of modules at each level of abstraction: how many of each for namespaces, classes per namespace, functions per class?
number of interfaces per module at each level of abstraction
the distance between interdependent modules
the amount of information transmitted between modules
Social complexity is characterized by
how many layers in the hierarchy?
the number of modules at each level of hierarchy: how many of each for organizations, teams-per-organization, and people-per-team
the number of interfaces at each level of hierarchy. A person talks to how many other people? A team coordinates with how many other teams?
the distance between interdependent modules. When two teams have to coordinate, are they in the same parent organization? Or separate organizations?
the amount of information transmitted between modules. A short email, a long telephone call, or multiple in-person meetings?
[Claim] When refactoring a system (whether software redesign or re-oganization of people roles and titles) that lacks redundancies, the complexity of the software+social is conserved if the functionality remains the same. Here conservation means productivity and efficiency are unchanged.
Even with the characterizations above and the definitions below I don't see how to quantify complexity in software-social systems as a scalar number or as a vector. I don't see how to rank complexity to enable comparison of different systems, or how the complexity of a system changes over time.
Definitions
In the characterizations above I used words that need explanation.
Definition of System
System: "an interconnected set of elements that is clearly organized in a way that achieves something"
source: "Thinking in Systems" by D. Meadows
In other words, a set of components + interactions that accomplish a goal. This applies to software, hardware, and social settings.
Definition of Distance within a System
Distance within a system of software is characterized by
services, which are decomposed into
namespace or package, which are decomposed into
class, which are decomposed into
methods or functions, which are decomposed into
lines
Distance in hierarchical social systems is characterized by
organization
team
person
Definition of Modularity
Modularity is defined as having these characteristics
The tables above illustrate that there are nuances to what "modularity" means in software. Modularity can improve flexibility for re-use but incurs cognitive load of the relationships and dependencies.
Definition of Complexity
Once you write a line of software it becomes legacy code and technical debt is incurred. Similarly in social settings, as soon as one person engages with another (cooperatively or competitively) there are decisions to be made about the interaction. The best you can aim for is to be aware of both the available trade-offs and the consequences of your actions when altering the system.
Obvious = relationship between cause and effect is obvious to system participants
Complicated = relationship between cause and effect requires analysis
Complex = relationship between cause and effect is clear only in hindsight
Chaotic = no relationship between cause and effect
When a software developer is exploring a software that is novel to them, the code can seem complex or complicated. After they have internalized the design and the reasoning for the design the same code base can seem obvious. Complexity is not a static characterization.
Similarly for interactions of people, a person new to an organization may initially feel overwhelmed, but after forming relationships and understanding the history views the organization as obvious.
Symptoms of complexity from Chapter 2 of "A Philosophy of Software Design" by Ousterhout
Change amplification: a simple change requires changes in many different places.
Cognitive load: the developer needs to learn a lot to complete a task.
Unknown unknowns: it is not obvious which pieces of code need to change to complete a task.
The point of decomposition is to make future work easier.
principle: "classes should be deep." Deep classes enable information hiding. The interface to the class is the cost; the functionality of the class is the benefit. Here "interface" includes signatures, side effects, and dependencies. The goal of abstraction is to maximize the ratio of functionality to interface.
Sometimes the cost of an interface can exceed the benefit of the functionality. For example, the number of characters for calling the interface may exceed the character count of enacting the function.
Smaller classes are easier to grok, but proliferation of classes increases the interfaces (and thus cost). "Classes are good" does not mean "more classes are better."
principle: "the common case should be simple"
principle: exceptions should be defined out of existence.
Tactical approach is to get some code working, accepting shortcuts to get the result. These shortcuts compound. Complexity isn't one mistake; complexity is hundreds or thousands of mistakes over a long duration. Because complexity doesn't happen all at once, the incremental mistakes are hard to notice. Observation: Software developers who are "tactical tornados" are rewarded by management because they appear productive in the short term. There's no recognition of the technical debt being created. Observation: "what's the smallest change I can make?" is a local optimization that often harms the global optimum.
The strategic approach is have the goal be "great design." Not just at the beginning, but with every change to code. This is slower compared to tactical approach but enables future productivity.
claim: The crossover in ROI might be something like 6 months for a project. Tactical can get results faster initially, but after 6 months making small changes gets more expensive than having used a strategic approach.
I recently started exploring how to migrate the Physics Derivation Graph website from the JSON-based backend (v7) to Neo4j (v8). For v7 I have a working JSON-based backend+frontend, and for v8 I have a working Neo4j-backend with minimal frontend (e.g., no nginx, no authentication, no documentation pages).
I'm aware of two ways to carry out the migration:
hard cut-over: port all existing front-end capabilities (nginx, authentication, documentation) and backend capabilities {render d3js, render tex, render pdf, render graphviz} to v8, then switch public site from v7 to v8
slow transition: add Neo4j to existing v7 JSON-based site and then incrementally transition site features like {render d3js, render tex, render pdf, render graphviz}
The "slow transition" has the advantage of not having to port the front-end capabilities. The downside is that the two codebases (v7, v8) exist concurrently and I have to refactor each backend feature on the live site.
I don't have a good sense for how much work porting front-end capabilities (the hard cut-over option) involves.
In the process of trying the "slow transition" I am getting lost in the code dependencies and which complexity is essential versus accidental. Refactoring requires the ability to distinguish essential versus accidental complexity.
The usual way to guide refactoring is to rely on tests of features. If I change some code and a feature breaks, then the essential complexity wasn't met. If I change something and all tests pass, then the assumption is that my change reduced accidental complexity. (This also relies on full test coverage and the assumption that tests are correlated with necessary capabilities.)
A different way to distinguish essential versus accidental complexity is to enumerate roles, use cases per role, and the requirements of each use case. The requirements are the essential complexity. Then the code I write is in response to a specific requirement.
In practice, as a developer my natural instinct is to enact the minimum amount of change based on my local perspective. To break out of this local minimum I need the context of tracing the roles to use cases to requirements. For each change to the software! That's asking a lot.
Writing out all the roles, uses cases, and requirements turns out to be pretty messy. This corresponds with the amount of code needed to support the Physics Derivation Graph.
Prior to upgrading from Ubuntu 20 to 22 LTS I was able to SSH from my local laptop to a remote VPS using the command
ssh -v username@IPaddress
After the upgrade I got
ssh -v username@IPaddress
OpenSSH_9.7p1, LibreSSL 3.3.6
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 21: include /etc/ssh/ssh_config.d/* matched no files
debug1: /etc/ssh/ssh_config line 54: Applying options for *
debug1: Authenticator provider $SSH_SK_PROVIDER did not resolve; disabling
debug1: Connecting to IPaddress [IPaddress] port 22.
debug1: connect to address IPaddress port 22: Operation timed out
ssh: connect to host IPaddress port 22: Operation timed out
status updates after ~30 hours of work over the course of 1 weekend.
made good progress with text input to web UI for Neo4j property graph back-end.
new user workflow: instead of user providing expressions during specification of step, now expressions are separate. (Not clear how well this will work for actually use.)
I'm happy with the rewrite of the back-end since it makes the design more robust and flexible.
I've learned enough Cypher to feel comfortable continuing with this path. I'm also somewhat confident (with no basis in experience) that switching to a different property graph is feasible without having to rewrite the front-end and controller).
Developer workflow
My "how do I develop the code" steps have evolved to include Black and mypy.
After making changes to the code I format using Black and then use type hinting:
make black_out
docker run --rm -v`pwd`:/scratch --entrypoint='' -w /scratch/ property_graph_webserver mypy --check-untyped-defs webserver/app.py
To launch the web UI,
make black_out; docker ps | grep property | cut -d' ' -f1 | xargs docker kill; date; make up
I added Selenium tests to speed up the UI validation.
Near-term plans
There's a lot to do, so in roughly priority order (from most important near-term to will-get-to-later)
Convert Latex expressions to Sympy
Check the dimensionality and consistency of expressions using sympy
Provide feedback to user on invalid inputs using dimensionality checks and symbol consistency
Check steps using SymPy. Currently in the v7 code base the file "validate_steps_sympy.py" takes Latex as input, converts to SymPy, then validates step. I think the step validation should be done using pure SymPy? (rather than taking Latex as input).
Lean: Explorer how to include lean derivations per step. Can I get lean to run on my computer?
Add API support to enable curl interactions.
This would improve command line testability of workflows
Write more selenium tests.
This assumes the web UI is stable.
A Physics derivation has steps and expressions. Expressions are composed of symbols and operations.
Symbols (e.g., x) can be constant or variable, real or complex. Symbols do not have arguments.
Operations are distinct from symbols because they require one or more symbols to operate upon. For example, +, ^, determinant(), etc.
Within the concept of symbols there are distinct categories: scalar, vector, matrix. The distinction is that they have different properties. A vector has a dimension, a matrix has 2 dimensions.
Since vectors can contain variables (e.g., \vec{a} = [x,y,z] ) and matrices can contain scalars, then we need a new boolean property for symbols: is_composite, and a new numeric property for symbols: dimensions.
When "dimension"=0, the symbol is a scalar. Then is_composite is false. When "dimension"=1, the symbol is a vector. Then is_composite is either false or true. If true then the symbol has two or more edges with other (scalar) symbols. When "dimension"=2, the symbol is a matrix. Then is_composite is either false or true. If true then the symbol has four or more edges with other (scalar) symbols.
However, this accommodation isn't sufficient. We can say "matrix A = matrix B" and consider the matrices as symbols. However, a quantum operator (e.g., bit flip) is a matrix that operates on a state -- an argument is required. There is an equivalence to the Hamiltonian ("H") and the matrix representation. Therefore the matrix is not a symbol as previously defined.
While the expression "matrix A = matrix B" is valid, " + = + " is not. The difference is that "+" is not a composite symbol.
"H = matrix B" is a valid expression even though "H" is an operator.
Therefore the distinction between symbol and operators is misleading. The schema for symbols should be
latex
name_latex
description_latex
requires_arguments (boolean)
if requires_arguments=true (operator: +, /, ^, determinant), then
argument_count (e.g., 1,2,3)
else requires_arguments=false (variable or constant or operator: a, x, H), then
dimension (e.g., 0,1,2)
if dimension = 0 (aka scalar: "size=1x1" and "is_composite=false") then
dimension_length
dimension_time
dimension_luminosity
dimension_mass
if dimension = 1 (aka vector) then
is_composite = false or true
orientation is row xor column xor arbitrary
if row or column,
size is arbitrary xor definite
if definite, "2x1" xor "1x2" xor "3x1" xor "1x3" ...
Initially the Physics Derivation Graph documented expressions as Latex. Then SymPy was added to support validation of steps (is the step self-consistent) and dimensionality (is the expression self-consistent?).
Recently I learned that Lean could be used to prove each step in a derivation. The difference between a Computer Algebra System (e.g., SymPy) and Lean is whether "a = b --> a/b = 1" is a valid step -- it isn't when b is zero. Lean catches that; SymPy does not.
While Lean proofs sound like the last possible refinement, there are two additional complications to account for not addressed by Lean.
Challenge: Bounded ranges of applicability
In classical mechanics the relation between momentum, mass, and velocity is "p = m v". That hold when "v << c". Near the speed of light we need to switch to relativistic mass,
m = m_{rest} / sqrt{1-((v^2)/(c^2))}.
The boundary between "v << c" and "v ~ c" is usually set by the context being considered.
One response for users of Lean would be to always use the "correct" relativistic equation, even when "v << c." A more conventional approach used by Physicists is to use
p = m v, where v << c
then drop the "v << c" clause and rely on context.
Challenge: Real versus Float versus experimental characterization
Lean forces you to characterize numbers as Real or Integer or Complex. This presents a problem for numerical simulations that have something like a 64 bit float representation.
In thermodynamics we assume the number of particles involved is sufficiently large that we focus on the behavior of the ensemble rather than individual particles. The imprecision of floats is not correct, but neither is the infinite precision assumed by Real numbers.
Example applications of Lean proofs needing bounds on values
Math doesn't have convenient ways of indicating "finite precision, as set by the Plank scale." The differential element used in calculus cannot actually go to zero, but we use that concept because it works at the scales we are used to.
Physicists make simplifying assumptions that sometimes ignore reality (e.g., assuming continuous media when particles are discrete). Then again the assumption that particles are discrete is also a convenient fiction that ignores the wavefunction of quantum mechanics.
Lean can be used to prove derivations in classical mechanics, but to be explicit about the bounds of those proofs we'd also need to indicate "v << c" and "assume space is Euclidean."
For molecular dynamics, another constraint to account for is "temperature << 1E10 Kelvin" or whatever temperature the atoms breaks down into plasma.
Distinguishing the context of (classical mechanics from quantum) and (classical from relativistic) and (conventional gas versus plasma) seems important so that we know when a claim proven in Lean is applicable.
In Physics there are some assumptions that form a dichotomy:
is the speed of light constant or variable?
is the measure of energy discrete or continuous?
In the dichotomy of assumptions, one of the two assumptions is reflective of reality, and the other is an oversimplification. The oversimplification is related to reality by assumptions, constraints, and limits.
(I define "oversimplification" as the extension of useful assumptions to incorrect regions.)
Another case where oversimplification is the link between domains is quantum physics and (classical) statistical physics. Quantum particles are either Fermions (odd half integer spin) or Bosons (integer spin), but that is practically irrelevant for large ensembles of particles at room temperature. The aspects that get measured at one scale (e.g., particle velocity) are related to but separate from metrics at another scale (e.g, temperature, entropy). Mathematically this transition manifests as the switch from summation to integration.
So what?
This is a new-to-me category of derivations which span domains. What constitutes a domain is set by the assumptions that form the boundaries, and oversimplification is how to cross the boundaries.
What boundaries should the Physics Derivation Graph transgress? What oversimplifications are adjacent?
The evidences of dissonance (e.g, Mercury’s perihelion based on Newtonian gravitation versus relativity, the Deflection of Starlight; source) are not relevant for bridging domains. They are illustrations of the oversimplification.
"by expressing quantum mechanics in phase space (the same ambit as for classical mechanics), the Weyl map facilitates recognition of quantum mechanics as a deformation (generalization) of classical mechanics, with deformation parameter ħ/S, where S is the action of the relevant process. (Other familiar deformations in physics involve the deformation of classical Newtonian into relativistic mechanics, with deformation parameter v/c; or the deformation of Newtonian gravity into general relativity, with deformation parameter Schwarzschild radius/characteristic dimension.)
Classical expressions, observables, and operations (such as Poisson brackets) are modified by ħ-dependent quantum corrections, as the conventional commutative multiplication applying in classical mechanics is generalized to the noncommutative star-multiplication characterizing quantum mechanics and underlying its uncertainty principle."
