Mathematics is meaningless
TL;DR: Mathematics only captures formal relationships between abstract symbols and cannot serve as a proper foundation for physics
In modern physics, there seems to be an overreliance on mathematics. Physical theories are first and foremost formulated in terms of mathematical language, and new ideas mainly stem from exploring new mathematical ideas (e.g. a new group, a formal generalization, addressing some mathematical problem and so on). There is, however, a fundamental problem in using mathematics as the starting point for a physical theory: mathematics is meaningless.
The general idea is that mathematics captures relationships between symbols, without regard to what the symbols stand for. For example, if we write the equation \(a = b c\), the only thing we are saying is that the quantity \(a\) is the product of the two quantities \(b\) and \(c\). That equation becomes Newton’s second law \(F = m a\) if the first symbol represents force, the second mass and the third acceleration. That same equation becomes Ohm’s law \(V = R I\) if the first symbol represents the voltage across a conductor, the second its resistence and the third the current. That same equation becomes the definition of R-value \(T = R_{val} \, \phi_q\) if the first symbol represents the temperature drop across an insulator, the second its R-value and the third the heat flow. Mathematically, these are the same and this is exactly why the theory of linear equations is so general and useful: it does not care what branch of science it is applied to.
The word “meaningless” may seem harsh, but it is usually used by mathematicians and philosophers. One of the most influential mathematicians, David Hilbert, said, “Mathematics is a game played according to certain simple rules with meaningless marks on paper.” One of the most influential philosophers in the foundations of mathematics, Bertrand Russell, said, “It is essential not to discuss whether the first proposition is really true, and not to mention what the anything is, of which it is supposed to be true.” Even if one does believe that mathematics is about real mathematical entities that do exist in some Platonic sense, these objects are not the physical ones we describe in physics. In that sense, mathematics is always physically meaningless.
The naive view would be that mathematics is about numbers, but this is too restrictive. The idea of order relation, for example, can be generalized to a surprising degree. If one does take the set of integers, or the set of real numbers, one uses the symbol \(\leq\) to describe whether one number is smaller than or equal to another. This relationship has three properties: it is reflexive (\(a \leq a\), a number is less than or equal to itself), antisymmetric (if \(a \leq b\) and \(b \leq a\) then \(a=b\)) and transitive (if \(a \leq b\) and \(b \leq c\) then \(a \leq c\)). If we forget that the symbols we are comparing are numbers and that \(\leq\) is comparing their magnitude, we can find a lot of other objects that follow the same rules.
For example, \(a\) and \(b\) could represent objects in the real world, and \(\leq\) could tell us whether an object is part of another. This relation would satisfy the same properties because: an object is part of itself (reflexive); if object \(a\) is part of \(b\) and \(b\) is part of \(a\) then \(a\) and \(b\) are the same object (antisymmetric); if \(a\) is part of \(b\) and \(b\) is part of \(c\), then \(a\) is part of \(c\) (transitive). Similarly \(a\) and \(b\) could represent subsets of a bigger set, and \(\leq\) could tell us whether the second set contains all the elements of the first. Yet another example, \(a\) and \(b\) could represent statements, and \(\leq\) could tell us whether the first statement implies the second (i.e. if the first is true, then the second must be true as well). The mathematical framework of order theory can describe all these cases precisely because it does not care whether the symbols represent numbers, objects, sets or statements.
In general, mathematics captures only the formal relationships between objects, without knowing what the objects are and whether they should obey those rules to begin with. Mathematics knows the rules of everything but the meaning of nothing. As such, using mathematics to find new physical ideas is like creating new recipes by combining the old ones without knowing what the ingredients are. Mathematics alone cannot provide a solid foundation for physical theories because it doesn’t know what is it talking about.