The new edition of Feynman's QED: The Strange Theory of Light and Matter

I was invited to write an introduction to the new edition of Feynman's classic book on quantum electrodynamics. For those interested, here is the introduction.

The story of how we came to know light makes for one gripping drama, complete with twists and turns and reversals of fortune.

The photon is the most visible of all elementary particles: place yourself in a dusty room with one small window open on a sunny day and watch a multitude of the little buggers hurrying across the room. Newton quite naturally thought that light consists of a stream of particles ("corpuscles") but already he had some doubts; even in the 17th century the diffraction of light could be readily observed. Eventually, diffraction and other phenomena appeared to show without doubt that light is an electromagnetic wave. That monument of 19th century physics, Maxwell's equations of electromagnetism, formulated light entirely as a wave. Then Einstein came along and explained the photoelectric effect by postulating light as the sum of little packets ("quanta") of energy. Thus were the word photon and the quantum theory of light born. (Here I will not digress and recall Einstein's famous discomfort with quantum mechanics even though he helped at its birth.) Meanwhile, from the 1920's through the 1940s physicists worked out the quantum behavior of matter ("atoms") thoroughly. Thus, it was all the more puzzling that the quantum behavior of light and its interaction with electrons resisted the efforts of the best and the brightest, notably Paul Dirac and Enrico Fermi. Physics had to wait for three young men, Feynman, Schwinger, and Tomonoga, filled with optimism and pessimism as the case may be from their experiences of the World War, to produce the correct formulation of quantum electrodynamics aka QED.

As probably every reader of the book knows, Richard Feynman (1918-1988) was not only an extraordinary physicist, but also an extraordinary figure, a swashbuckling personality the likes of which theoretical physics has not seen before or hence. Occasionally theoretical physicists would while away an idle moment comparing the contributions of Feynman and Schwinger, both nice Jewish boys from New York and almost exact contemporaries. This senseless discussion serves no purpose, but it is a fact that while Julian Schwinger was a shy and retiring person (but rather warm and good-hearted behind his apparent remoteness) Dick Feynman was an extreme extrovert, the stuff of legends. With his bongo drums, showgirls, and other trappings of a carefully cultivated image enthusiastically nurtured by a legion of idolaters, he is surely the best-loved theoretical physicist next to Einstein.

The brilliant Russian Lev Landau famously had a logarithmic scale for ranking theoretical physicists with Einstein on top. It is also well known that Landau moved himself up half a step after he formulated the theory of phase transitions. I have my own scale of fun, on which I place theoretical physicists I know either in person or in spirit. Yes, it is true: most theoretical physicists are dull as dish water and rank near minus infinity on this logarithmic scale. I would place Schrödinger (about whom more later) on top, but Feynman would surely rank close behind. I can't tell you where I land on my own scale, but I do try to have as much fun as possible, limited by the amount of talent and resources at my disposal.

But what fun Feynman was! Early in my career, Feynman asked me to go to a nightclub with him. One of Feynman's colleagues told me that the invitation showed that he took me seriously as a physicist, but while I was eager to tell Feynman my thoughts about Yang-Mills theory he only wanted my opinion on the legs of the dancing girls on stage. Of course, in the psychology of hero worship, nobody gives two hoots about some bozo of a physicist playing drums and liking showgirls. So all right, my scale is really fun times talent --- Landau's scale with fun factored in, with the stock of Einstein falling and Landau rising (he played some good pranks until the KGB got him.)

Now some thirty years after that night club visit I felt honored that Ingrid Gnerlich of Princeton University Press asked me to write an introduction to a new edition of Feynman's famous book "QED: The Strange Theory of Light and Matter." First a confession: I had never read this book before. When this book came out in 1985 I had just finished writing my first popular physics book "Fearful Symmetry" and I more or less adopted a policy of not reading other popular physics books for fear of their influencing my style. Thus, I read the copy Ingrid sent me with fresh eyes and deep appreciation. I enjoyed it immensely, jotting down my thoughts and critiques as I went along.

I was wrong not to have read this book because it is not a popular physics book in the usual sense of the phrase. When Steve Weinberg suggested in 1984 that I write a popular physics book and arranged for me to meet his editor in New York he gave me a useful piece of advice. He said that most physicists who wrote such books could not resist the urge of explaining everything while the lay reader only wanted to have the illusion of understanding and to catch a few buzz words to throw around at cocktail parties.

I think that Weinberg's view, though somewhat cynical, is largely correct. Witness the phenomenal success of Hawking's "A Brief History of Time" (which I have not read in accordance to the policy I mentioned earlier.) One of my former colleagues here at the University of California, a distinguished physicist who now holds a chair at Oxford, once showed me a sentence from that book. The two of us tried to make sense of it and failed. In contrast, I want to assure all the puzzled readers that every sentence in this book, though seemingly bizarre to the max, makes sense. But you must mull over each sentence carefully and try hard to understand what Feynman is saying before moving on. Otherwise, I guarantee you that you will be hopelessly lost. It is the physics that is bizarre, not the presentation. After all, the title promises a "strange theory."

Since Feynman is Feynman, he chose to go totally against the advice Weinberg gave me (advice which I incidentally also did not follow completely; see my remark below regarding group theory). In the acknowledgement, Feynman decried popular physics books as achieving "apparent simplicity only by describing something different, something considerably distorted from what they claim to be describing." Instead, he posed himself the challenge of describing QED to the lay reader without "distortion of the truth." Thus, you should not think of this book as a typical popular physics book. Neither is it a textbook. A rare hybrid it is instead.

To explain what kind of book this is, I will use Feynman's own analogy, somewhat modified. According to Feynman, to learn QED you have two choices: you can either go through 7 years of physics education or read this book. (His figure is a bit of an overestimate; these days a bright high school graduate with the proper guidance could probably do it in less than 7 years.) So you don't really have a choice, do you? Of course you should choose to read this book! Even if you mull over every sentence as I suggest you do, it should still take you less than 7 weeks, let alone 7 years.

So how do these two choices differ? Now comes my version of the analogy: a Mayan high priest announced that for a fee he could teach you, an ordinary Joe in Mayan society, how to multiply two numbers, for example 564 by 253. He made you memorize a 9 by 9 table and then told you to look at the two digits farthest to the right in the two numbers you have to multiply, namely 4 and 3, and say what is in the 4th row and 3rd column. You said 12. Then you learned that you should write down 2 and "carry" 1, whatever that meant. Next you were to say what is in the 6th row and 3rd column, namely 18, to which you were to add the number you were carrying. Of course, you had to spend another year learning how to "add." Well, you got the idea. This is what you would learn after paying tuition at a prestigious university.

Instead, a wise guy named Feynman approached you, "Shh, if you know how to count, you don't have to learn all this fancy stuff about carrying and adding! All you've got to do is to get hold of 564 jars. Then you put into each jar 253 pebbles. Finally, you pour all the pebbles out onto a big pile and count them. That's the answer!"

So you see, Feynman not only teaches you how to multiply, but also gives you a deep understanding of what the high priests and their students, those people soon to have Ph.D.'s from prestigious universities, are doing! On the other hand, if you learn to multiply Feynman's way you couldn't quite apply for a job as an accountant. If your boss asks you to multiply big numbers all day long, you would be exhausted, and the students who went to High Priest University would leave you in the dust.

Having written both a textbook ("Quantum Field Theory in a Nutshell" henceforth to be referred to as Nutshell) and two popular physics books (including "Fearful Symmetry" henceforth Fearful) I feel that I am quite qualified to address your concerns about what kinds of books to read. By the way, Princeton University Press, the publisher of this book, publishes both Nutshell and Fearful.

Let me divide the readers of this introduction into three classes: (1) students who may be inspired by this book to go on and master QED, (2) intelligent lay persons curious about QED, and (3) professional physicists like myself.

If you are in (1), you will be so incredibly inspired and fired up by this book that you will want to rush out and start reading a textbook on quantum field theory (and it might as well be Nutshell!) By the way, these days QED is considered a relatively simple example of a quantum field theory. In writing Nutshell I contend that a truly bright undergrad would have a good shot at understanding quantum field theory, and Feynman would surely agree with me.

But as in the analogy, reading this book alone will in no way turn you into a pro. You have to learn what Feynman referred to as the "tricky, efficient way" of multiplying numbers. In spite of Feynman's proclaimed desire to explain everything from scratch, he noticeably runs out of steam as he goes on. For example, on page 89 and in figure 56, he merely describes the bizarre dependence of P(A to B) on the "interval I" and you just have to take his word for it. In Nutshell, this is derived. Similarly for the quantity E(A to B) described in the footnote on page 91.

If you are in (2), persevere and you will be rewarded, trust me. Don't rush. Even if you only get through the first two chapters you would have learned a lot. Why is this book so hard to read? We could go back to the Mayan analogy: it is as if you are teaching someone to multiply by telling him about jars and pebbles but he doesn't even know what a jar or a pebble is. Feynman is bouncing around telling you about each photon carrying a little arrow, and about how you add up these arrows and multiply them, shrinking and rotating them. It is all very confusing; you can't afford even the slightest lapse in attention. Incidentally, the little arrows are just complex numbers (as explained in a footnote on page 63) and if you already know about complex numbers (and jars and pebbles) the discussion might be less confusing. Or perhaps you are one of those lay readers described by Weinberg as typical, who are satisfied with "the illusion of understanding something." In that case, you may be satisfied with a "normal" popular physics book. Again the Mayan analogy: a normal popular physics book would burden you neither with 9 by 9 tables and carrying, nor with jars and pebbles. It might simply say that when given two numbers the high priests have a way of producing another number. In fact, editors of popular physics books insist that authors write like that in order not to scare away the paying public (more below).

Finally, if you are in (3), you are in for a real treat. Even though I am a quantum field theorist and know what Feynman is doing, I still derived great pleasure from seeing familiar phenomenon explained in a dazzlingly original and unfamiliar way. I enjoyed having Feynman explaining to me why light moves in a straight line or how focusing lens really works (on page 58: "A trick can be played on Nature" by slowing light down along certain paths so the little arrows all turn by the same amount!)

Shh, I will tell you why Feynman is different from most physics professors. Go ask a physics professor to explain why in the reflection of light from a pane of glass it suffices to consider reflection from the front surface and the back surface only. Very few would know the answer (see page 104). It is not because physics professors lack knowledge, but because it has never even occurred to them to ask this question. They simply study the standard textbook by Jackson, pass the exam, and move on. Feynman is the pesky kid who is forever asking why why WHY!

With three classes of readers (the aspiring student, the intelligent layperson, the pro) there are also three categories of physics books (not in one-to-one correspondence): textbooks, popular books, and what I might call "extra-difficult popular physics books." This book is a rare example of the third category, in some sense intermediate between a textbook and a popular book. Why is this third category so thinly populated?

Because "extra-difficult popular physics books" scare publishers half to death. Hawking famously said that every equation halves the sale of a popular book. While I do not deny the general truth of this statement I wish that publishers would not be so easily frightened. The issue is not so much the number of equations, but whether popular books could contain honest presentation of difficult concepts. When I wrote Fearful, I thought that to discuss symmetry in modern physics it would be essential to explain group theory. I tried to make the concepts accessible by the use of little tokens: squares and circles with letters inside them. But the editor compelled me to water the discussion down repeatedly until there was practically nothing left, and then to relegate much of what was left to an appendix. Feynman, on the other hand, had the kind of clout that not every physicist writer would have.

Let me return to Feynman's book with its difficult passages. Many of the readers of this book would have had some exposure to quantum physics. They may be legitimately puzzled for example by the absence of the wave function that figures so prominently in other popular discussion of quantum physics. Quantum physics is puzzling enough --- as a wit once said, "With quantum physics, who needs drugs?" Perhaps the reader should be spared further head scratching. So let me explain.

Almost simultaneously but independently, Erwin Schrödinger and Werner Heisenberg invented quantum mechanics. To describe the motion of an electron for example, Schrödinger introduced a wave function governed by a partial differential equation, now known as the Schrödinger equation. In contrast, Heisenberg mystified those around him by talking about operators acting on what he called quantum states. He also famously enunciated the uncertainty principle, which physically states that the more accurately one were to measure say the position of a quantum particle the more uncertain becomes one's knowledge of its momentum, and vice versa.

The formalisms set up by the two men were manifestly different, but the bottom-line result for any physical processes they obtained always agreed. Later, the two formalisms were shown to be completely equivalent. Today, any decent graduate student is expected to pass from one formalism to the other with facility, employing one or the other according to which one is more convenient for the problem at hand.

Six years later, in 1932, Paul Dirac suggested, in a somewhat rudimentary form, yet a third formalism. Dirac's idea appeared to be largely forgotten until 1941 when Feynman developed and elaborated this formalism, which became known as the path integral or sum over history formalism. (Physicists sometimes wonder whether Feynman invented this formalism completely ignorant of Dirac's work. Historians of physics have now established that the answer is no. During a party at a Princeton tavern, a visiting physicist named Herbert Jehle told Feynman about Dirac's idea and apparently the next day Feynman worked out the formalism in real time in front of the awed Jehle. See the 1986 article by S. Schweber in Reviews of Modern Physics.)

It is this formalism that Feynman tries hard to explain to you in this little book. For example, on page 43, when Feynman adds all those arrows he is actually integrating (which of course is calculus jargon for summing) over the amplitudes associated with all possible paths the photon could follow in getting from the point S to the point P. Hence the term “path integral formalism.” The alternative term “sum over history” is also easy to understand. Were the rules of quantum physics relevant to affairs on the macroscopic human scale, then all alternative histories, such as Napoleon triumphing at Waterloo or Kennedy dodging the assassin's bullet, would be possible and each history is associated with an amplitude that we are to sum over (“summing over all those little arrows.”)

It turns out that the path integral, regarded as a function of the final state, satisfies the Schrödinger equation. The path integral is essentially the wave function. Hence the path integral formalism is completely equivalent to the Schrödinger and Heisenberg formalisms. In fact, the one textbook that explains this equivalence clearly was written by Feynman and Hibbs. (Yes, Feynman has also authored textbooks, you know, those boring books that actually tell you how to do things efficiently, like "carrying" and "adding". Also yes, you guessed correctly that Feynman's textbooks are often largely written by his co-authors.)

Since the Dirac-Feynman path integral formalism is completely equivalent to the Heisenberg formalism, it most certainly contains the uncertainty principle. So Feynman's cheerful dismissal of the uncertainty principle on page 56 is a bit of an exaggeration. At the very least one can argue over semantics: what did he mean saying that the uncertainty principle is not "needed"? The real issue is whether or not it is useful.

Theoretical physicists are a notoriously pragmatic lot. They will use whichever method is the easiest. There is none of the mathematicians' petulant insistence on rigor and proof. Whatever works, man!

Given this attitude, you may ask, which of the three formalisms, Schrödinger, Heisenberg, and Dirac-Feynman, is the easiest? The answer depends on the problem. In treating atoms for example, as the master himself admits on page 100, the Feynman diagrams "for these atoms would involve so many straight and wiggly lines and they'd be a complete mess!" The Schrödinger formalism is much easier by a long shot and that is what physicists use. In fact, for most "practical" problems the path integral formalism is almost hopelessly involved, and in some cases downright impossible. I once even asked Feynman about one of these apparently impossible cases and he had no answer. Yet beginning students using the Schrödinger formalism easily solve these apparently impossible cases!

Thus, which formalism is best really depends on the physics problem, so that theoretical physicists in one field, atomic physics for example, might favor one formalism, while those in another, high energy physics for example, might prefer another formalism. Logically then, it may even happen that, as a given field evolves and develops, one formalism may emerge as more convenient than another.

To be specific, let me focus on the field I was trained in, namely high energy or particle physics, which is also Feynman's main field. Interestingly, in particle physics the path integral formalism for a long time ran a distant third in the horse race between the three formalisms. (By the way, nothing says that there could be only three. Some bright young guy could very well come up with a fourth!) In fact, the path integral formalism was so unwieldy for most problems that by the late 1960s it almost fell into complete obscurity. By that time, quantum field theory was almost exclusively taught using the canonical formalism, which is merely another word for the Heisenberg formalism, but the very word "canonical" should tell you which formalism was held in the highest esteem. Just to cite one case history I happen to know well, I never heard of the path integral during my student days, even though I went to two reasonably reputable universities on the east coast for my undergraduate and graduate studies. (I mention the east coast because for all I know it was taught intensively in an eastern enclave in Los Angeles.) It was not until I was a postdoc at the Institute for Advanced Study before I, and most of my colleagues, were first alerted to the path integral formalism by a Russian paper. Even then, various authorities expressed doubts about the formalism, saying for example that it could not account for the chiral anomaly (the reader need not be concerned with what that is.)

Ironically, it was Feynman himself who was responsible for this deplorable state of affairs. What happened was that students easily learned the "funny little diagrams" (such as those on page 116) invented by Feynman. Julian Schwinger once said rather bitterly that "Feynman brought quantum field theory to the masses," by which he meant that any dullard could memorize a few "Feynman rules", call himself or herself a field theorist, and build a credible career. Generations learned Feynman diagrams without understanding field theory. Heavens to Betsy, there are still university professors like that walking around!

But then almost incredibly --- and perhaps this is part of the Feynman mystique that gave his career an almost magical aura, in the early 1970s starting largely with that Russian paper I just mentioned, the Dirac-Feynman path integral made a roaring comeback so that it quickly became the dominant way to make progress in quantum field theory.

Notice that this is also what makes Feynman such an extraordinary physicist: the "battle for the hearts and minds" I just described was between the crowd using Feynman diagrams versus a younger crowd using Feynman path integrals. I hasten to add that the word "battle" is a bit strong: nothing prevents a physicist from using both. I did, for one.

I believe that my recent textbook Nutshell is one of the few that employ the path integral formalism right from the beginning, in contrast to older textbooks that favor the canonical formalism. I started the second chapter with a section titled "The professor's nightmare: a wise guy in the class". In the spirit of all those apocryphal stories about Feynman I made up a story about a wise guy student and named him Feynman. The path integral formalism was derived by the rather Zen procedure of introducing an infinite number of screens, drilling an infinite number of holes in each screen, thus ending up with no screen. But as in the Mayan priesthood analogy, after this Feynmanesque derivation, I had to teach the student how to actually calculate ("carrying" and "adding") and for that I had to abandon the apocryphal Feynman and go through the detailed Dirac-Feynman derivation of the path integral formalism, introducing such technicalities as "the insertion of 1 as a sum over a complete set of bras and kets." Technicality is what you do not get by reading Feynman's books!

Incidentally, in case you are wondering, the bras have nothing to do with the philandering Dick Feynman. They were introduced by the staid and laconic Paul Dirac as the left half of a bracket. Dirac is himself a legend: I once sat through an entire dinner with Dirac and others without him uttering more than a few words.

I chuckled a few times as Feynman got in some sly digs at other physicists. For example, on page 132 he dismissively referred to Murray Gell-Mann, the brilliant physicist and Feynman's friendly rival at Caltech, as a "great inventor." Going somewhat against his own cultivated image, he then deplored on page 135 the general decline of physicists' knowledge of Greek, knowing full well that Gell-Mann not only coined the neologism "gluon" but is also an accomplished linguist.

I also liked Feynman self-deprecatory remarks that are part and parcel of his image. On page 149, when Feynman spoke of "some fool physicist giving a lecture at UCLA in 1983" some readers might not realize that Feynman was speaking of himself! Although this is indeed part of the image, I find it refreshing as theoretical physicists become increasingly hierarchical and pompous in our time. The Feynman whom I knew, and I emphasize that I did not know him well, would surely not like this trend. He once caused a big fuss trying to resign from the National Academy of Sciences.

Referring back to the three classes of potential readers I described above, I would say that those in (2) and (3) will enjoy this book enormously, but the book was secretly written for those in (1). If you are an aspiring theoretical physicist, I urge you to devour this book with all the fiery hunger you feel in your mind, and then go on to learn from a quantum field theory textbook how to actually "carry".

Surely you can master quantum field theory. Just remember what Feynman said: "What one fool can understand, another can." He was referring to himself, and to you!