Book of the Universe
The Book of the Universe
Ingmar de Boer, v. 3

Philosophy is written in this grand book, which is continually open before our eyes (I call it the Universe) but cannot be understood, if one does not first learn to understand the language, and knows the signs, in which it is written. It is written in the language of mathematics, and the signs are triangles, circles, and other geometrical figures, without which means it is humanly impossible to understand one word of it; without these, one is wandering through a dark labyrinth in vain. (Galileo Galilei, in Il Saggiatore, p. 285 in the 1623 ed., Engl. IdB)
 La Filoſofia è ſcritta in queſto grandiſſimo libro, che continuamente ci ſta aperto innanzi agli occhi (io dico l’ Univerſo) ma non ſi può intendere, ſe prima non s'impara a intender la lingua, e conoſcer i caratteri, ne' quali è ſcritto. Egli è ſcritto in lingua matematica, e i caratteri ſon triangoli, cerchi, ed altre figure Geometriche, ſenza i quali mezzi è impoſſibile intenderne umanamente parola; ſenza queſti è un aggirarſi vanamente per un oſcuro laberinto.
Introduction
In his 1623 work Il Saggiatore (The Assayer), Galilei responds to various points of critique on his earlier work. The cited fragment is considered an important statement in the history of science, since it is thought to be the first time in history when the universe is seen as essentially "mathematical". Let us sum up the picture Galilei has painted for us.
 The universe is compared metaphorically to a book in which philosophy is encoded.
 The book can only be understood knowing its language, that of mathematics.
 The signs of the language are geometrical figures.
Due to this metaphore, this fragment seems to reflect at the same time two different opinions about mathematics, the first being the idea that mathematics is a language, and the second that mathematics is inherently part of reality itself. Further, we could argue that if mathematics is a language, it cannot be an inherent part of nature, since language originates in the human mind. It is part of culture instead of nature, and it depends entirely on collective agreements. It is therefore not part of "the world outside". Language is not discovered, but invented, or perhaps defined. Vice versa, if mathematics is embedded in reality, it cannot be a language, since in that case it originates in the outside world, and is discovered, instead of invented by man.
The Mathematical Universe
In his 2014 work Our Mathematical Universe, as well as the 2007 article The Mathematical Universe, the sympathetic Max Tegmark is putting forward the same idea as Galilei. His theory is presented as the mathematical universe hypothesis (MUH), stating in brief, that "our external physical reality is a mathematical structure". (see p. 207 of the book) The problem here, is that the term "mathematical" is used loosely (and not metaphorically), not only as the language of mathematics, as the scientific area of research, but also as the order, the regularities found in nature. Structures in reality are strictly speaking not mathematical but physical structures, and the scientific discipline which studies them is called physics. For example, Tegmark speaks of symmetry as a mathematical property (p. 265), and of course in everyday speech we would call it that, but strictly speaking it is not a property of mathematics, but of physics. According to Tegmark, the definition of mathematics should be taken "broad enough" to encompass the whole of physical reality. (p. 271) He then goes even a step further when he says that everything which exists in mathematics should also exist in reality. This seems easily to be falsified by creating a mathematical fantasy which does not correspond to anything in any reality. However, if we predefine any mathematical reality as ontological reality, it becomes unfalsifiable, introducing an even greater problem. Again, it cannot be both at the same time.
To support his argument he defines a hypothesis, the external reality hypothesis (ERH), which is "accepted by most but not all physicists". This is of course not completely true, if it were only because no physicist had heard of this hypothesis before Tegmark defined it himself. Further, it seems strange that "most but not all physicists" would "accept" a hypothesis without any proof: hypotheses are not things to be accepted, but things to be proven or disproven. Obviously this is again a loose way of formulating.
In mainstream western philosophy there has been an ancient debate about the existence or nonexistence of external reality. In physics however, the existence of external reality is not generally put up for discussion. One reason for this is, not that physicists are accepting a hypothesis without proof, but that physicists usually leave this kind of problems to philosophers, and that philosophy's judgement on the "mindonly" world is not very favourable. Solipsism, as it is called there, is often seen as an untenable or particularly unproductive worldview, and for that reason it does not attract a lot of interest from philosophers today.
Another reason for not doubting external reality in physics, is related to the essence of natural science itself. The larger problem which is the reason of existence of natural sciences is that external reality is only known to us by the grace of the senses, and not by mathematical discovery or imagination. This is why we have to measure things in the external world and design experiments to discover relations between what we have measured. These relations are strictly speaking, again, not mathematical but physical, however there is perhaps a case to be made for calling these both mathematical and physical. Summarising: if the universe was really "mathematical", there would not have existed any form of natural science, neither would there have been any need for it.
The World Described by a Subset of Possible Descriptions
Let us return to the point of view that mathematics would be a language. What can be said in most languages is much more than what we can perceive, or vice versa: reality corresponds to only a subset of what we can describe. For example, we can state that Edward is the son of Henry and Jane as well as of Albert and Victoria, but it is clear that if we are speaking of the same Edward, this cannot be true. Nevertheless, as a sentence there is nothing wrong with it. The language of mathematics is in that respect not different from natural languages, as it is also able to describe much more than what could exist in reality.
A telling example in natural science is the mathematical invention of complex numbers. In technical applications complex numbers seem to function well, and they are even very important in many areas of physics and engineering, but we do not have any idea what they refer to in reality. A "physicalist" would perhaps argue that as long as we are not able to investigate their reality by means of the methods of natural science, they cannot be real. They do exist in mathematics but do not have any counterpart in reality. For this reason physicists have investigated if the use complex numbers (or at least their imaginary parts) might be eliminated from physics.
Platonism in Mathematics
Following the 1934 lecture by Paul Bernays Sur le platonisme dans les mathematiques, the term Platonism is used (in a strict or less strict sense) in western philosophy to indicate the idea that mathematical entities exist in reality. In fact, in the English version of the article produced from the lecture text, Bernays formulates it as follows:
 [...] the tendency of which we are speaking consists in viewing the objects as cut off from all links with the reflecting subject. Since this tendency asserted itself especially in the philosophy of Plato, allow me to call it "platonism".
The term does not do justice to the great philosopher, since his thoughts about mathematics were probably almost the exact opposite of what Bernays is suggesting here. According to the actual Platonists, mathematics belonged primarily to the "world of ideas", which is not an objective world which is "cut off from the reflecting subject". Around 2000 years later, Immanuel Kant also interpreted this the same way: in chapter I.2.1.2.3 of his Kritik der Reinen Vernunft, he associates the world of ideas, the noumenal, with the domain of reason.
On the other hand, we know the discoveries of mathematics are not only subjective in nature. Mathematics is not based on introspection in the sense that its discoveries are only of value to the person who observes them in his own subjective inner world. This duality between "inner" knowledge and objectivity makes the status of mathematics as a science hard to understand, at least at first glance. We may think that the progression of neuroscience can shed definitive light on this.
The Structure of Reality
Perhaps things will become clearer if we investigate the role of mathematics in more detail here. Despite Galilei's sincere argument, actual triangles or circles are not found in nature. The language of geometry, of triangles and circles—or anything else we have learnt for that matter—may serve as a tool to model natural phenomena, but they cannot be building blocks of the phenomenal world, since they do not exist in that world. If one considers triangles and circles to be parts of a language, of geometry, and not of reality, then mathematics should also be considered to be a type of language, or perhaps a set of modelling tools, but not the structure of reality. The structure of the universe is therefore not "mathematical". Mathematics is a way for us to perceive, analyse, or define the structure of reality. Certainly, we should not be calling something mathematics/mathematical from now on, which has been called physics/physical for centuries, and think that that in itself will solve any fundamental problems.
In a sense, we could even consider mathematics the only way to describe the stable connections between quantities, that we call laws of nature. Without language or modelling tools, we cannot describe nature. This means that we can only discover the laws of nature if they satisfy certain conditions. That is one aspect of why it seems that mathematics is a surprisingly appropriate language to describe nature. We can only perceive that reality which is "mathematical", because we see our way of understanding things reflected in everything we perceive. We could not perceive nature if it was "unmathematical", and it could not exist to us. In the words of Ludwig Wittgenstein's Tractatus:

4.113 Philosophy sets limits to the much disputed sphere of natural science.
4.114 It must set limits to what can be thought; and, in doing so, to what cannot be thought. It must set limits to what cannot be thought by working outwards through what can be thought.
4.115 It will signify what cannot be said, by presenting clearly what can be said.

4.113 Die Philosophie begrenzt das bestreitbare Gebiet der Naturwissenschaft.
4.114 Sie soll das Denkbare abgrenzen und damit das Undenkbare. Sie soll das Undenkbare von innen durch das Denkbare begrenzen.
4.115 Sie wird das Unsagbare bedeuten, indem sie das Sagbare klar darstellt.