The symbolic AI community is increasingly trying to embrace machine learning
in neuro-symbolic architectures, yet is still struggling due to cultural
barriers. To break the barrier, this rather opinionated personal memo attempts
to explain and rectify the conventions in Statistics, Machine Learning, and
Deep Learning from the viewpoint of outsiders. It provides a step-by-step
protocol for designing a machine learning system that satisfies a minimum
theoretical guarantee necessary for being taken seriously by the symbolic AI
community, i.e., it discusses "in what condition we can stop worrying and
accept statistical machine learning." Unlike most textbooks which are written
for students trying to specialize in Stat/ML/DL and willing to accept jargons,
this memo is written for experienced symbolic researchers that hear a lot of
buzz but are still uncertain and skeptical. Information on Stat/ML/DL is
currently too scattered or too noisy to invest in. This memo prioritizes
compactness, citations to old papers (many in early 20th century), and concepts
that resonate well with symbolic paradigms in order to offer time savings. It
prioritizes general mathematical modeling and does not discuss any specific
function approximator, such as neural networks (NNs), SVMs, decision trees,
etc. Finally, it is open to corrections. Consider this memo as something
similar to a blog post taking the form of a paper on Arxiv.Comment: 12 pages of main contents, 29 pages in total. It could also serve as
an accompanying material for Latplan paper. (arXiv:2107.00110) v2: rewrote
the general ELBO derivation without Prolog. v3: significantly extended the
Bayesian reasoning section in the appendix, with several proofs for conjugate
priors. v4+: errata fi