| A |
Democritus extended the atomic concept as far as it could go, claiming that not just matter, but everything else from colour to the human soul must also consist of atoms. These atoms were indivisible but had different shapes and could combine in a variety of ways to form the substances of the world. He saw creation as the natural consequence of the ceaseless whirling motion of atoms in space. Atoms would collide and spin, forming larger aggregations of matter.
These ideas were soon rivalled by the very different philosophies of Aristotle from the school of Plato, who believed that matter was infinitely divisible and that nature was constructed from perfect symmetry and geometry. According to Empedocles substance was composed of four continuous elements; Earth, Air, Fire and Water. Only with the Islamic Caliphates who studied the earlier Greek philosophers, did the atomistic theory hold out during the middle ages. Al-Razi of Persia is credited with an atomistic revival in the ninth century but Aristotle's physics remained the dominant doctrine in European philosophy until the seventeenth century.
In the 1660s Robert Boyle, a careful chemist and philosopher proposed a corpuscular theory of matter to explain behaviour of gases such as diffusion. According to Boyle there was only one fundamental element, all corpuscles would be identical. Different substances would be constructed by combining the corpuscles in different ways. The theory was based as much on the alchemist's belief in the existence of a philosopher's stone which could turn lead into gold, as it is was on empirical evidence. Newton built on the corpuscular theory. He saw the corpuscles as units of mass and introduced the laws of mechanics to explain their motion.
In 1808 the atomic theory was again resurrected by a school teacher and amateur scientist by the name of John Dalton. He discovered a law of partial pressures of gases which revealed how gases of equal volume contribute pressures in nearly integer ratios. He concluded that these were ratios of atomic weights which were a characteristic of indivisible atoms. This would also explain chemical composition and the nature of the chemical elements. Amedeo Avogadro developed the molecular theory and his law that all gases at the same temperature, pressure and volume contain the same number of molecules even though their weights are different. By the mid nineteenth century the number of molecules in a volume of gas could be measured. Maxwell and Boltzmann went on to explain the laws of thermodynamics through the statistical physics of molecular motion. The atomic theory was having unprecedented success in explaining a wide variety of physical phenomena.
Despite this indirect evidence, positivists led by Ernst Mach remained sceptical about the kinetic theory. They argued that since atoms could not be directly observed they are no more than metaphysical constructs with no basis in reality. The pressure of such disputes was too much for Boltzmann who took his own life in 1906. Ironically, Einstein had provided what would transpire to be the clinching evidence for atoms just the previous year. In the early eighteenth century, a biologist Robert Brown had observed random motion of particles suspended in gases. Einstein explained that this Brownian motion could be seen as direct experimental evidence of molecules which were jostling the particles with their own movements. In 1956 the field ion microscope made it possible to form images of individual atoms for the first time.
How far has modern physics gone towards the ideal of Democritus that everything should be composed of discrete units?
The story of light parallels that of matter. The Greeks saw an atomistic theory of light as the explanation of light rays. In the Arabic world of the middle ages Al-hazen used a ballistic theory of light to explain reflection. Newton extended Boyle's corpuscular theory to light even though such a supposition had no empirical foundation at that time. Everything he had observed and much more was later explained by Maxwell's theory of Electromagnetism in terms of waves in continuous fields. It was Planck's Law and the photoelectric effect which later upset the continuous theory. These phenomena could only be explained in terms of light quanta. Today we can detect the impact of individual photons on CCD cameras even after they have travelled across most of the observable universe from the earliest moments of galaxy formation.
Those who resisted the particle concepts had, nevertheless, some good sense. Light and matter, it turns out, are both particle and wave at the same time. This paradox is explained mathematically as a consequence of quantum field theory but the interpretation remains unintuitive and mysterious.
As it turned out, the atomic theory of Dalton was a long way short of the end of the road for divisibility. The atom was split and broken down into its constituent particles, and they were in turn further divided. The way we now describe the composition of matter is no longer so simple. When a neutron is observed to decay spontaneously into a proton, neutron, electron and neutrino we do not suppose that those four particles were parts of the neutron which broke apart. Particles can transform and interact in a way which is not simply division and recombination of immutable parts. Physicists continue their journey into the heart of matter, and the final picture has not yet been seen.
Two hundred years after Newton, James Clerk Maxwell unified electricity, magnetism and light into one theory of electromagnetism. This unification was the result of a series of experiments starting in 1820 when Hans Christian Oersted observed that an electric current deflected a compass and Andre Ampere measured the corresponding reaction force on a current in a magnetic field. Above all it was Michael Faraday who appreciated the significance of these results and devised the experiments which would unveil the unity of nature. He showed that a moving magnet could induce a current in a wire and also noticed that a magnetic field could change the polarisation of light passing through a medium. Faraday is regarded as possibly the greatest experimental physicist who ever lived and he proposed the idea of force lines but he never used equations to describe his theories. It was only when Maxwell applied mathematics to the problem that the full power of electromagnetic unification was realised.
The atomic theory was the other important unification step of the nineteenth century. Prior to 1808 chemistry was little more than a catalogue of chemicals and their reactions, although the distinction between elements and compounds had been recognised by Antoine Lavoisier in 1786. The molecular theory was also already part of the kinetic theory of gases when John Dalton proposed that molecules were composed of immutable atoms. By 1869 Dmitri Mendeleyev had laid out the periodic table of the elements in order of atomic weights. By the end of the nineteenth century most everyday observations could be accounted for in terms of well-known physics, and some scientists thought that little remained to be understood. They failed to see the lack of unity which remained in their theories. Mass, energy, space, time, charge, the ether and atoms were the basic constituents whose behaviour followed the laws of mechanics, electromagnetics, gravity, chemistry, electricity and thermodynamics. Other sciences such as biology and astronomy could have been regarded as reducible to these terms but the case for vitality in biology still held sway and astronomy was still a realm apart.
Even then there were other new phenomena, and unexplained enigmas were appearing: By 1900 the electron, X-rays and nuclear radiation had been discovered. Experiments had failed to detect the ether and electromagnetism and thermodynamics could not explain black body radiation. The spectral lines in light already seen by Fraunhofer in 1814, the anomalous perihelion shift of Mercury discovered by Le Verrier in 1859 and the photoelectric effect of Hertz in 1887, were all indications of future revolutions. That is easy for us to see now, but at the turn of the century these things might just have easily been accounted for by making small adjustments to known physics. Many physicists were unprepared for what was to come, but not all. At the dawn of the new century Henri Poincaré wrote that there was a whole new world of which none had expected the existence but that further progress would show how these complete the general unity.
Our greatest lesson of the twentieth century is what Poincaré foresaw, that the universe is governed by a profound unity of physical law. The revelation began with the special relativity of Poincaré and Einstein which Minkowski recognised as a unification of space and time into a single space-time geometry. Mass and energy were then also seen as equivalent, or at least interchangeable. In the same decade the Planck-Einstein theory of light quanta brought together electromagnetics and thermodynamics. Then Einstein unified space-time and gravity into one theory of general relativity and the atomic theory was reduced to quantum mechanics by Bohr, Heisenberg, Schrödinger and others. The quantum theory also produced an unexpected unification of particles and waves. Later, when Dirac brought together special relativity and quantum mechanics he predicted anti-matter particles which were found shortly after. At the same time as all this unification, new things like the nuclear forces, new particles, superfluidity, and quantum spin were being found but they were all part of the new physics. The total number of fundamental concepts needed to account for nature had diminished drastically.
By the end of the first half of the century the theory of quantum electrodynamics was complete. The world was then recovering from the second world war. Physicists had served their part, for better or for worse, by developing radar and the atomic bomb. No doubt it was by way of repayment, or the hope of further military spin-offs, that they were granted funds to build the large accelerators which were to dominate the discoveries in physics of the following decades. Suddenly there was a new wealth of particles and properties to explain. In 1960 physics was a messy catalogue of particle properties, but the lesson had already been learnt and the search for unity prevailed again. Yang-Mills gauge theories were the key to understanding the forces. By the mid-seventies the quark theory, quantum chromodynamics and the electro-weak force were part of a standard model of particle physics.
At the end of the twentieth century physics is able to explain much more than everyday observations. It can explain just about every fundamental observation that we have been capable of making up to now, from the laboratory to the cosmos. The last quarter of the century has been a tough time for experimenters. They were impotent in their search for new phenomena and could do no more than verify the standard model in ever greater detail. That is not to say that experiments made no contribution to knowledge since the mid-seventies. While the standard model has been verified, many new theories which were advanced have been ruled out through negative results, allowing the theorists to concentrate their efforts on those which remain.
But the main impetus which has been pushing forward the front of physics over the last twenty years has come from a belief in complete unity. According to conventional wisdom among physicists, the process of unification will continue until all physics is unified into one neat and tidy theory. There is no a priori reason to be so sure that this must happen. It is quite possible that physicists will always be discovering new forces, or finding new layers of structure in particles, without ever arriving at a final theory. It is quite simply the unified nature of the laws of physics as we currently know them, the lesson of the twentieth century, that inspires the belief that we are getting closer to that end.
After physicists discovered the atom, they went on to discover that it was composed of electrons and a nucleus, then that the nucleus was composed of protons and neutrons, then that the protons and neutrons were composed of quarks. Should we expect to discover that quarks and electrons are made of smaller particles? This is possible but there are reasons to suppose not. Firstly there are far fewer particles in the standard model than there ever were at higher levels. Secondly, their interactions are described by a clean set of gauge bosons through renormalisable field theories. Composite interactions, such as pion exchange, do not take such a tidy form. These reasons in themselves are not quite enough to rule out the possibility that quarks, electrons and gauge bosons are composite but they reduce the number of ways such a theory could be constructed. In fact all viable theories of this type which have been proposed are now all but ruled out by experiment. There may be a further layer of structure but it is likely to be different. It is more common now for theorists to look for ways that different elementary particles can be seen as different states of the same type of object. The most popular candidate for the ultimate theory of this type is superstring theory, in which all particles are just different vibration modes of very small loops of string.
Physicists construct particle accelerators which are like giant microscopes. The higher the energy they can produce, the smaller the wavelength of the colliding particles and the smaller the distance scale they probe. In this way, physicists can see the quarks inside protons, not through direct pictures but through scattering data. They have already examined quarks at a scale of 10-19 metres and they still look pointlike. Such resolution is impressive given that atoms have a typical size of 10-10 metres and nucleons have structure on the scale of 10-15 metres. Suppose you have a cannon ball about 10 centimetres in diameter in your hand. Imagine you scale it up until it is as big as the Earth (a factor of 108). The bumps and scratches on the surface would have become mountain ranges and great ravines. As you walked over the surface you could look down at the ground and would see that it is made of atoms scaled up to the size of marbles 1 or 2 centimetres across. Each atom would be a hazy cloud of electrons around the tiny nucleus which appears as just a point in the centre.
Now scale one of those atoms again by the same factor. It would now be about the size of Pluto. The nucleus will have expanded to a huge jumble of nucleons, each the size of a house but appearing as a fuzz of quarks. If you could now stop one of the electrons or quarks in the atom and look at it closely with the naked eye, you would be seeing it on the scale which today's biggest accelerators have probed, so we know that it would still look like a point. Despite this impressive achievement we have only gone half way towards the smallest scale. If the superstring theory is right and electrons and quarks have no structure until you see them on the string scale, it will be necessary to scale them up twice again by the same factor before they become visible as little loops of string. The atom, now scaled up by a factor of 1032, would then be about a million light years across. The scale of inner space is as impressive as the scale of outer space.
In the first decade of the 21st century new accelerator experiments at CERN will probe beyond the electro-weak scale. There is some optimism that new physics will be found but nothing is certain. After that, experimental particle physics may become more difficult. There is a limit to how much funding for larger accelerators can be found, even with many nations clubbing together. Perhaps other observational clues will come from cosmic rays and big bang cosmology. Perhaps experimenters will get lucky and find a better way to accelerate particles. If they could have a wish granted it might be the discovery of a stable charged elementary particle with a 1000 times the mass of the proton. It could then be produced in quantity and accelerated to much higher energy. Alternatively they might ask for a new form of stable matter which can be built into superdense substances. Even with such luck there is a long way to go before reaching the scale of grand unification, but ingenuity and the unexpected should never by underestimated in experimental physics.
In any case, that empirical route is just the low road, and there is an alternative high road which the theorists can take while the lower remains blocked. Progress may come from the mathematical search for greater unity. The electromagnetic and nuclear forces are now only partially unified. They still have separate coupling strengths in the standard model. There are also three generations of quark-lepton matter quadruplets and that need to be explained. Perhaps there should be unification of the gauge bosons of the force fields and the fermionic matter fields. Above all gravity must be brought together with the other forces. That will require a unification of general relativity and quantum mechanics. By searching through the mathematical possibilities for new forms of unity, physicists may be able to bypass the huge gap in energy between current day experiments and the higher unification scales. Ironically, as a result of such endeavours, we may already know more about physics at distance scales of 10-36 metres than we do at scales of 10-24 metres.
The problem which they face is to put together general relativity and quantum mechanics into one self consistent theory. The difficulty is that the two parts seem to be incompatible, both in concept and in practice. A direct approach, attempting to combine general relativity and quantum mechanics, while ignoring conceptual differences, leads to a meaningless quantum field theory with unmanageable divergences. Conceptually, it is the nature of space and time, seen differently from each side, which present the fundamental differences. There have, in fact, been many attempts to create a theory of quantum gravity. From some of these it appears that the combination of general relativity and the quantum theory will also be a unification of much more. It will probably require all four forces and the matter fields to be brought together. It may also require a deeper unification of space-time and matter. If this is true, a complete theory of quantum gravity will then be the realisation of Descartes's visionary dream. It will be the final step on the long road of unification which he foresaw.
Einstein's special relativity was the culmination in 1905 of the work of many physicists such as Lorentz and Poincaré. Mechanics and electrodynamics were placed in a new kinematic framework in which space and time were no longer absolute. When Minkowski described a geometric formulation of special relativity in which space and time were combined into a single space-time continuum, at first Einstein did not like it. Soon he changed his mind as he recognised that this geometric way to understand relativity was more easy to generalise than his original mechanical approach. He wanted to extend relativity to include gravity. His genius is demonstrated by the way in which he was able to perceive the correct principles which were needed and follow their consequences to the right conclusion.
General relativity is based on two fundamental principles: The principle of relativity which states that all basic laws of physics should take a form which is independent of any reference frame, and The principle of equivalence which states that it is impossible to distinguish (locally) the effects of gravity from the effects of being in an accelerated frame of reference.
Einstein struggled with the consequences of these principles for several years, constructing many thought experiments to try to understand what they meant. He had already recognised the value of the equivalence principle in 1907. Finally he learnt about Riemann's mathematics of curved geometry and in 1912 realised that a new theory could be constructed in which the force of gravity was a consequence of the curvature of space-time.
In constructing that theory, Einstein was not significantly influenced by any experimental result which was at odds with the Newtonian theory of gravity. He knew of the anomalous precession of the perihelion of Mercury and hoped that a new theory might explain it but there is no route to develop general relativity directly from such an observation. He also knew, however, that Newtonian gravity was inconsistent with his theory of special relativity and he knew there must be a more complete self consistent theory. A similar inconsistency now exists between quantum mechanics and general relativity and, even though no experimental result is known to violate either theory, physicists now seek a more complete theory in the same spirit.
By 1915 Einstein's work was complete. The force of gravity was now a consequence of geometrodynamics; the dynamic geometry of space-time. The equations for the gravitational field are complicated but are an almost unique consequence of the relativity principles which require that they must be independent of any co-ordinate system. Einstein calculated the motion of Mercury in his theory and found that the relativistic corrections to the Newtonian prediction correctly accounted for its anomalous motion. He then predicted that star-light passing the sun would be deflected by twice the Newtonian amount. Arthur Eddington measured this deflection on a South American expedition to observe a solar eclipse in 1919. When he announced to the world that the result agreed with the prediction of general relativity, Einstein became a household name synonymous with "genius".
In the decades that have followed Einstein's discovery, a number of other experimental confirmations of general relativity have been found, and geometrodynamics has become the cornerstone of cosmology. There still remains a possibility that it may not be accurate on very large scales, or under very strong gravitational forces. There are, however, no alternative theories with the force of elegance found in general relativity. The fortuitous discovery by Hulse and Taylor of a binary pulsar in 1974, made it possible to test and verify general relativistic effects to very high precision. Still, the theory is sure to break down finally under the conditions which are believed to have existed at the big bang where quantum gravity effects were important.
One of the most spectacular predictions of general relativity is that a dying star of sufficient mass will collapse under its gravitational weight into an object so compressed that not even light can escape its pull. Such collapsed objects were designated "black holes" by John Wheeler in 1967 and the picturesque term has stuck. Astronomers now have a growing list of celestial objects which they believe are black holes because of their apparent high density and because of evidence of matter apparently falling silently through the event-horizon. The accuracy of Einstein's theory may be stringently tested again in the near future when gravitational wave observatories such as LIGO come on-line to observe such catastrophic events as the collisions between black holes.
Unlike general relativity which was essentially the work of one man, the quantum theory required major contributions from Bohr, Einstein, Heisenberg, Schrödinger, Dirac and many others, before a complete theory of quantum electrodynamics was formulated. In practical terms, the consequences of the theory are more far reaching than those of general relativity. Applications such as transistors and lasers are now an integral part of our lives and, in addition, the quantum theory allowed us to understand chemical reactions and many other phenomena.
Despite such spectacular success, confirmed in ever more detail in high energy accelerator experiments, the quantum theory is still criticised by some physicists who feel that its indeterministic nature and its dependency on the role of observer suggest an incompleteness. For others the major task is to combine general relativity and quantum mechanics. Opinions differ as to how much revision of quantum mechanics is required to achieve it. Perhaps quantum mechanics is more fundamental than general relativity or perhaps it is the other way round. The answers lie in the realms of ultra-high energy physics, well beyond what can be attained experimentally with known techniques. This leaves us with theory as the only means of moving ahead for the time being at least.
At first thought it might seem ridiculous to suppose that we can invent valid theories about physics at high energies before doing experiments. However, theorists have already demonstrated a remarkable facility for doing just that. The standard model of particle physics was devised in the 1960s by theoretical physicists. It described the physics of energies several orders of magnitude beyond what had been observed before. Experimentalists have spent the last three decades verifying it. The reason for this success is that physicists recognised the importance of certain types of symmetry and self-consistency conditions in quantum field theory which led to an almost unique model for physics up to the electro-weak unification energy scale, with only a few parameters such as particle masses to be determined.
The situation now is a little different. Experimentalists are about to enter a new scale of energies and theorists do not have a single unique theory about what can be expected there. They do have some ideas, in particular it is hoped that supersymmetry may be observed, but we will have to wait and see.
Despite these unknowns there are other more general arguments which tell us things about what to expect at higher energies. When Planck initiated the quantum theory he recognised the significance of fundamental constants in physics, especially the speed of light (known as c), Boltzmann's constant (known as k) and Planck constant (known as h). Scientists and engineers have invented a number of systems of units for measuring lengths, masses, temperature and time, but they are entirely arbitrary and must be agreed by international convention. Planck realised that there should be a natural set of units in which the laws of physics take a simpler form. The most fundamental constants, such as c, k and h would simply be equal to one unit in that system.
If one other suitable fundamental constant could be selected, then the units for measuring mass, length and time would be determined. Planck decided that Newton's gravitational constant (known as G) would be a good choice. Actually there were not many other constants, such as particle masses known at that time, otherwise his choice might have been more difficult. By combining c, h, k and G, Planck defined a system of units now known as the Planck scale. In 1899 he wrote that it is possible to give units for length, mass, time and temperature which retain their meaning for all time and all cultures, even extraterrestrial ones. He calculated that the Planck unit of length is very small, about 10-35 metres. To build an accelerator which could see down to such lengths would require energies about 1016 times larger than those currently available. The Planck scale is not very good for practical engineering, partly because the units are mostly either too small or too big compared with everyday quantities. More importantly, it is not possible to make accurate enough measurements using Planck units because it would be necessary to measure the mass of an object by measuring its gravitational pull on other objects. However, Planck units are very convenient for physicists studying quantum gravity because the values of the constants c, h, k and G are equal to one and can be left out of the equations.
Physicists have since sought to understand what the Planck scale of units signifies. One possibility is that at the Planck scale all the four forces of nature, including gravity, are unified. Physicists who specialise in general relativity have a different idea. In 1955 John Wheeler argued that when you combine general relativity and quantum mechanics you will have a theory in which the geometry of space-time is subject to quantum fluctuations. He computed that these fluctuations would become significant if you could look at space-time on length scales as small as the Planck length. Sometimes physicists talk about a space-time foam at this scale but we do not yet know what it really means. For that we will need the theory of quantum gravity.
Without really knowing too much for certain, physicists guess that at the Planck scale all forces of nature are unified and quantum gravity is significant. It is at the Planck scale that they expect to find the final and completely unified theory of the fundamental laws of physics.
It seems clear that to understand quantum gravity we must understand the structure of space-time at the Planck length scale. In the theory of general relativity space-time is described as a smooth continuous manifold but we cannot be sure that this is correct for very small lengths and times. We could compare general relativity with the equations of fluid dynamics for water. They describe a continuous fluid with smooth flows in a way which agrees very well with experiment. Yet we know that at atomic scales, water is something very different and must be understood in terms of forces between molecules whose nature is completely hidden in the ordinary world. If space-time also has a complicated structure at the tiny Planck length, way beyond the reach of any conceivable accelerator, can we possibly hope to discover what it is?
If you asked a group of mathematicians to look for theories which could explain the fluid dynamics of water, without them knowing anything about atoms and chemistry, then they would probably succeed in devising a whole host of mathematical models which work. All those models would probably be very different, limited only by the imagination of the mathematicians. None of them would correspond to the correct description of water molecules and their interactions. The same might be true of quantum gravity in which case there would be little hope of finding out how it worked without further empirical information. Nevertheless, the task of putting together general relativity and quantum mechanics together into one self consistent theory has not produced a whole host of different and incompatible theories. The clever ideas which have been developed have things in common. It is quite possible that all the ideas are partially correct and are aspects of one underlying theory which is within our grasp. It is time now to look at some of those ideas.
In this scheme, particles are a consequence of the field quantisation and are seen as less fundamental than the field waves out of which they appear. The particles carry spin in integer or half-integer multiples of Planck's constant. They may be spin zero, spin a half or spin one according to the type of field which is quantised. All the known fundamental fermions such as quarks and electrons are spin half. The gauge bosons which mediate the electromagnetic and nuclear forces are spin one. There are also thought to be Higgs particles which have spin zero but they have not yet been found in experiments. The interactions between these particles can be most easily worked out using a perturbation theory. The clearest form of this is a diagrammatic system which was worked out by Richard Feynman.
In principle it should be possible to apply the same quantisation methods to the gravitational field. It is necessary to first construct a system of non-interacting graviton particles which represent a zero order approximation to quantised gravitational waves in flat space-time. These hypothetical gravitons must be massless particles carrying spin two, because of the form of the gravitational field in general relativity. The next step is to describe the interactions of these gravitons using the perturbation theory. Feynman himself spent a significant amount of time trying to get it to work, but for gravity this simply cannot be done in the way that works for the Yang-Mills gauge fields. The calculations are plagued by infinite quantities which cannot be renormalised. The resulting quantum field theory is incapable of giving any useful result.
Because quantum gravity is an attempt to combine two different fields of physics, there are two distinct groups of physicists involved. These two groups form a different interpretation of the failure of the direct attack. The relativists say that it is because gravity cannot be treated perturbatively. To try to do so destroys the basic principles on which relativity was founded. It is, for them, no surprise that this should not work. Perturbation theory requires that you define a fixed approximate background and treat the full physics as if it was a perturbative deviation from there. The fixed background breaks the relativistic symmetry of general covariance. On the other hand, particle physicists say that if a field theory is non-renormalisable then it is because it is incomplete. The theory must be modified and new fields might have to be added to cancel divergences, or it may be that the observed fields are approximate composite structure of more fundamental constituent fields.
Particle physicists discovered that if the symmetry of space-time is extended to include supersymmetry, then it is necessary to supplement the metric field of gravity with other matter fields. Miraculously these fields led to cancellations of many of the divergences in perturbative quantum gravity. This has to be more than coincidence. At first it was thought that such theories of supergravity might be completely renormalisable. After many long calculations this hope faded. A strange thing about supergravity was that it works best in ten or eleven-dimensional space-time. This inspired the revival of an old theory from the 1920s called Kaluza-Klein theory, which suggests that space-time has more dimensions than the four obvious ones. The extra dimensions are not apparent because they are curled up into a small sphere with a circumference as small as the Planck length. This theory provides a means to unify the gauge symmetry of general relativity with the internal gauge symmetries of particle physics.
The next big step taken by particle physicists came along shortly after. Two physicists Michael Green and John Schwarz were looking at a theory which had originally been studied as a theory of the strong nuclear force but which was actually more interesting as a theory of gravity because it included spin-two particles. This was the new beginning of string theory. Combining string theory and supergravity to form superstring theory quickly led to some remarkable discoveries. A few string theories in ten dimensions were perfectly renormalisable and finite. This was exactly what they were looking for.
It seemed once again that the solution was near at hand, but nature does not give up its secrets so easily. The problem now was that there is a huge number of ways to apply Kaluza-Klein theory to the superstring theories. Hence there seem to be a huge number of possible unified theories of physics. The perturbative formulation of string theory makes it impossible to determine the correct way.
For a long time there seemed little hope of finding any solutions to the Wheeler-DeWitt equation. Then in 1986 Abhay Ashtekar found a way to reformulate Einstein's equations of gravity in terms of new variables. Soon afterwards a way was discovered to find solutions to the equations. This is now known as the loop representation of quantum gravity. Mathematicians were surprised to learn that knot theory was an important part of the concept.
The results from the canonical approach seem very different from those of string theory. There is no need for higher dimensions or extra fields to cancel divergences. Relativists point to the fact that a number of field theories which appear to be unrenormalisable have now been quantised exactly. There is no need to insist on a renormalisable theory of quantum gravity. On the other hand, the canonical approach still has some technical problems to resolve. It could yet turn out that the theory can only be made fully consistent by including supersymmetry.
As well as their differences, the two approaches have some striking similarities. In both cases they are trying to be understood in terms of symmetries based on loop like structures. It seems quite plausible that they are both aspects of one underlying theory. Other mathematical topics are common features of both, such as knot theory and topology. Indeed there is now a successful formulation of quantum gravity in three-dimensional space-time which can be regarded as either a loop representation or a string theory. A small number of physicists such as Lee Smolin are looking for a more general common theory uniting the two approaches.
Fields can be multiplied together event by event. Differential operators which act on the fields are defined using the continuous nature of the space-time co-ordinates. The equations of evolution for the fields are specified using these operations which ensure their causal and local nature. In the new approach fields are defined by their algebraic properties and space-time co-ordinates are ignored. Fields are any kind of mathematical structure which can be multiplied together and which can be operated on by some operators which obey rules analogous to those of differentiation, such as Leibniz rule for products.
If enough algebraic rules are applied the new type of fields will be equivalent to the old traditional definition for a space-time with some kind of topology. If the rules are allowed to differ then a more general structure than space-time is defined. The rule which is the most likely candidate for change is that fields should multiply together commutatively. This is analogous to the step taken in going from classical to quantum physics where observables are replaced by non-commuting observables. Now the same idea is used to define non-commutative geometry.
The technique can also be applied successfully to groups by generalising the algebraic properties of a function from the group to the real numbers. The result in this case is the discovery of quantum groups which have all the important algebraic properties of functions on a group except commutivity. Space-time structure can be derived from its group of symmetries in a way which can be generalised to quantum groups. The result is various forms of quantum space-time. The hope of this program is that general relativity and quantum field theories can also be generalised and that the results will not suffer from the infinite divergences which are the primary obstacle to a theory of quantum gravity.
Because of the stringent constraints that self consistency enforces, it is possible to construct thought experiments which provide strong hints about the properties a theory of quantum gravity has to have. There are two physical regimes in which quantum gravity is likely to have significant effects. In the conditions which existed during the first Planck unit of time in our universe, matter was so dense and hot that unification of gravity and other forces would have been reached. Likewise, a small black hole whose mass corresponds to the Planck unit of mass also provides a thought laboratory for quantum gravity.
Black holes have the classical property that the surface area of their event horizons must always increase. This is suggestively similar to the law that entropy must increase, and in 1972 it led Jacob Bekenstein to conjecture that the area of the event horizon of a black hole is in fact proportional to its entropy. If this is the case then a black hole would have to have a temperature and obey the laws of thermodynamics. Stephen Hawking investigated the effects of quantum mechanics near a black hole using semi-classical approximations to quantum gravity. Against his own expectations he discovered that black holes must emit thermal radiation in a way consistent with the black hole entropy law of Bekenstein.
This forces us to conclude that black holes can emit particles and eventually evaporate. For astronomical sized black holes the temperature of the radiation is minuscule and certainly beyond detection, but for small black holes the temperature increases until they explode in one final blast. Hawking realised that this creates a difficult paradox which would surely tell us a great deal about the nature of quantum gravity if we could understand it.
The entropy of a system can be related to the amount of information required to describe it. When objects are thrown into a black hole the information they contain is hidden from outside view because no message can return from inside. Now if the black hole evaporates, this information will be lost in contradiction to the laws of thermodynamics. This is known as the black hole information loss paradox.
A number of ways on which this paradox could be resolved have been proposed. The main ones are:
Assuming that something has not been missed out, which is a big assumption, we must conclude that the amount of entropy which can be held within a region of space is limited by the area of a surface surrounding it. This is certainly counterintuitive because you would imagine that you could write information on bits of paper and the amount you could cram in would be limited by the volume only. This is false because any attempt to do that would eventually cause a black hole to form. Note that this rule does not force us to conclude that the universe must be finite because there is a hidden assumption that the region of space is static.
If the amount of information is limited then the number of physical degrees of freedom in a field theory of quantum gravity must also be limited. Inspired by this observation, Gerard 't Hooft, Leonard Susskind and others have proposed that the laws of physics should be described in terms of a discrete field theory defined on a space-time surface rather than throughout space-time. They liken the way this might work to that of a hologram which holds a three-dimensional image within its two-dimensional surface.
Rather than being rejected as a crazy idea, this theory has been recognised by many other physicists as being consistent with other ideas in quantum gravity, especially string theory.
If Susskind is right, this solution to the information loss problem may have even stranger consequences. What happens in the case of an observer, Mr. X, who falls into a black hole. From his point of view he will pass through the event horizon without incident and continue to his gruesome fate at the black-hole singularity. Any knowledge and information he carries will stay with him till the end. To an outside observer, Miss Y, the situation must be different. Gravitational time dilation ensures that she will watch Mr. X slow down so much as he approaches the event horizon that he will never cross it. Eventually he will fade from her view but the information he carries must still be accessible. If Miss Y waits long enough the black hole will evaporate and the information will be returned in the radiation. At least it should be in principle even if it is too jumbled to be read in practice.
There is a conceptual difficulty which accompanies this situation. The course of events as witnessed by Mr. X is very different from that seen by Miss Y. If they are ever brought together in a court of law and asked to account for what happened to the information their stories will not be consistent. Mr X will claim he carried it to his cosmic grave where time ended for him but Miss Y will say that it never got past the event horizon and was brought back into the outside universe as the hole evaporated. The judge and jury will be forced to conclude that one of them was lying. This paradox is resolved by the simple fact that the two witnesses never can be brought back together. Presumably this must even be true if the black hole harboured a wormhole through to another universe through which Mr. X could escape his fate.
Susskind has called this the black hole complementarity principle in deference to Niels Bohr's complementarity principle of quantum mechanics. Just as there is no conflict between the dual properties of matter as both particle and wave because no observation brings them into contradiction, so too there is no conflict between the contrary observations of Mr. X and Miss Y. The implications of Susskind's principle may be even harder to contemplate than Bohr's. In ordinary quantum mechanics observers who can communicate freely should be able to agree what the probability of future events is. However, if one plans to take a swan dive into a black hole he may not agree on the most likely future events with his partner who plans to rest outside. This removes physics further from the conventional causal paradigms. The full implications may only be understood when we have a complete consistent theory which embraces the new complementarity.
Although there has been considerable progress on the problem of quantising gravity, it seems likely that it will not be possible to complete the solution without some fundamental change in the way we think about space-time. To face the quantum gravity challenge we need new insights and more new principles like those which guided Einstein to the correct theory of gravity.
At one point supergravity looked very promising as a theory which might unify all physics. At the time I was a student at Cambridge University where Stephen Hawking was taking up his position as the new Lucasian professor of mathematics. There was great anticipation of his inaugural lecture to take place on 29th April 1980. Even though I made a point of turning up early I found only standing room at the back of the auditorium. It was an exciting talk at which Hawking made some of his most quotable comments. He cautiously predicted that the end of theoretical physics was in sight. The goal might be achieved in the not-too-distant future, perhaps by the end of the century.
But early hopes faded as the perturbative calculations in supergravity became difficult and it seemed less likely that it defined a renormalisable field theory. There were other difficulties such as the problem of fitting in the distinction between left and right which we find in the weak force. Hawking pointed out himself that he was joining a list of physicists who had thought they were near the end. Faraday thought a unification of gravity and electromagnetism would lead to a complete theory but he could not detect any effect linking the two as he had with electricity and magnetism. After the rapid progress in the foundations of quantum mechanics in the 1920s Max Born told a meeting of scientists that physics would be over in six months. Einstein, in his later years, also thought that a unified theory was within reach. Those hopes were premature.
In 1985 The phrase "Theory of Everything" entered the minds of theoretical physicists. It came up in articles written for science magazines such as New Scientist and Science and later appeared in the title of a number of books. The discovery that set things going was that the heterotic superstring theory is finite in all orders of perturbation theory and has the potential to encompass all the known theories of particle physics and gravity too. In other words it provided potentially a unified theory of all the known underlying laws of physics.
It was not long before scientists from other disciplines and physicists too, started to question the validity of the claim that superstring theory was a theory of everything. For one thing it did not really make any testable predictions, leading some to retort that it was more like a theory of nothing. More to the point, they questioned whether any theory of physics could rightly be called a theory of everything. They were quite right.
The term Theory of Everything is a desperately misleading one. Physicists usually try to avoid it but the media apparently cannot help themselves. "Physicists on the verge of finding theory of everything." It makes too good a headline. If physicists find a complete unified set of equations for the laws of physics, then that would be a fantastic discovery. The implications would be enormous, but to call it a theory of everything would be nonsense.
For one thing, it would be necessary to solve the equations to understand anything. No doubt many problems in particle physics could be solved from first principles, perhaps it would be possible to derive the complete spectrum of elementary particles including their relative masses and the coupling constants of the forces which bind them. However, there would certainly be limits to the solvability of the equations. We already find that it is almost impossible to derive the spectrum of hadrons composed of quarks, even though we believe we have an accurate theory of strong interactions. In principle any set of well-defined equations can be solved numerically given enough computer power. The whole of nuclear physics and chemistry ought to be possible to calculate from the laws we now have. In practice computers are limited and experiments will never be obsolete.
Furthermore, it is not even possible to derive everything in principle from the basic laws of physics. Many things in science are determined by historical accident. The foundations of biology fall into this category. The final theory of physics will not tell us how life on Earth originated. The most ardent reductionist would retort that, in principle, it would be possible to derive a list of all possible forms of life from the basic laws of physics. Such justification is weak. No theory of physics is likely to answer all the unsolved problems of mathematics, chemistry, biology, astronomy or medicine.
Finally it must be said that even given a convincing unified theory of physics, it is likely that it would still have the indeterminacy of quantum mechanics. This would mean that no argument could finally lay to rest questions about paranormal, religion, destiny or other such things, and beyond that there are many matters of philosophy and metaphysics which might not be resolved, not to mention an infinite number of mathematical problems.
But string theorists never claimed that their work was applicable to any of these things. Steven Weinberg tried to clarify what it was all about in his 1988 book "Dreams of a Final Theory". Physicists, he argued, are seeking to take the last step of unification on a climb which started as least as far back as Newton. Those steps could lead us towards one "Final Theory" in which all the underlying laws of physics are unified. Weinberg's term "Final Theory" is actually not much better than "Theory of Everything". It suggests, to some, that the theory will mark the end of science and there will be no new theories after. Again, this is not what is meant. Finding the final laws of physics will be like arriving at the summit of the highest mountain. It is a special place from where you can see far, but getting there does not mean you have been everywhere.
In my youth I found time to explore the mountains of Scotland where I lived. Often as you climb one of those rounded peaks, you see ahead what appears to be the top. As you get closer you realise that it is a false summit with a further climb beyond. Sometimes there are several of them before you reach the true summit and at last take in the panoramic view, if the mist and rain have cleared. Approaching the final theory of physics seems to be a very similar experience. There have already been many false summits and again we see another ahead. A mountaineer always knows that there is a final summit and it can be reached if he has the courage to continue. Can physicists know that their summit is there too? Hawking feels that it is. After Cambridge the next time I had the opportunity to hear Hawking lecture was 17 years on at a conference for string theorists.
Hawking had never moved on from supergravity to string theory as other physicists had, until then. His liking for strings appeared to have improved when it was discovered in 1995 that string theories can be unified under a mysterious form of supergravity in 11 dimensions. Hawking must have felt that he had been vindicated in his prediction that supergravity was near the end. With a characteristic touch of humour he told us, "twenty years ago, I said there was a 50/50 chance that we would have a complete picture of the universe in the next twenty years. That is still my estimate today but the 20 years start now."
There are a few who are not so certain. John Taylor in his book "When the clock struck zero" argued that there could be an infinite structure of levels of physical law to find. No-one thinks that there will be a final theory of mathematics and if mathematics is so strongly reflected in physics why should there be a limit to its application? For what my opinion is worth, I too think we really are near the summit.