Hello, and welcome to the class! My name is Michael Frank, and this is the CISE special-topics course on the Physical Limits of Computing. This is a course that strikes at the core of the future of our civilization. We live in a very exciting time. The US economy is roaring along and is about to break a record for the longest period of economic expansion in its history. And the world economy overall has been growing very well lately also. Now, I'm not an economist, but I believe there's some level of consensus that a large part of the economic expansion of the last half-century can be attributed to the myriad advances in information-processing technology. The application of computer technology has facilitated all sorts of progress in nearly every industry. And the ever-increasing usefulness of computers has been enabled primarily by the rapid and continuing improvement in cost-efficiency of the raw, underlying computational devices - which for about the last 40 years have consisted of semiconductor-based electronic transistors.
In 1965, Gordon Moore, then of Fairchild Semiconductor, observed a trend of increasing performance in the first few generations of integrated-circuit technology. Knowing a few things about how this performance might continue to be improved, he predicted that in fact it would continue to improve at an exponential rate - with the performance per unit cost increasing by a factor of 2 every 18 months or so - for at least the next 10 years. Well, it turned out he was correct, but not only for the next 10 years - instead, the computer industry has remained remarkably on-track with his prediction throughout the entirety of the last 40 or 50 years. Raw computational power per unit cost has improved by an incredible factor of [?] since 1965, and as we know, this has enabled all sorts of amazing advances in communications, manufacturing, finance, and all manner of products and services. It has affected just about everything. And as you may know, Gordon Moore went on to found Intel, which is now one of the largest companies in the world, and makes the CPU chips for the majority of our ubiquitous computers.
Given these amazing recent trends, I think that at the present moment in human history, perhaps the most important, critical, overriding, question for the future of human technology and economic development is this: How long can Moore's law continue to hold? What are the ultimate limits, if any, to computing technology? And how will the technology need to change in order to improve as much as possible?
To answer these questions, we first have to understand why Moore's law has held true so far. What has enabled these amazing technical improvements? The vast majority of the improvement can be attributed to a single factor: miniaturization. What Gordon Moore was actually projecting back in 1965 was the number of transistors that could be manufactured on the surface of a single computer chip. As semiconductor manufacturing techniques improved, the transistors could be made smaller and smaller, and more and more cheaply. But unlike the case with other sorts of manufactured goods such as cars, airplanes, construction equipment, factory tools, etc., which have to be at least a certain size in order to serve their intended purpose, the amazing thing about transistors is that their performance at their task - information processing - continually improved as they got smaller. The smaller they got, the closer they could be together, so signals didn't have to travel as far, and it took less electrical energy to encode bits of data, power consumption decreased, and system reliability improved as more elements were integrated into a single part. And cost decreased because the bulk manufacturing methods that the industry uses - involving flashes of light and chemical baths - could pattern an entire chip at once, and so the more transistors you have per chip, the more you could make with given material resources in a single pass through the assembly line.
So, here we have an amazing thing: A product that seems to get better in every respect as you make it smaller and smaller, and what's more, you can make it smaller as you improve the fabrication process in a fairly methodical way. But, how long can this trend continue?
Well, it turns out there are a lot of fundamental physical limits that will ultimately come to bear on the shrinkage of transistors, and the improvement of computing technology in general.
There are of course lots of fairly near-term manufacturing concerns, such as how to manipulate light wavelengths small enough to etch out very tiny features - ultraviolet light is in use now, and the industry is trying to figure out how to use X-rays effectively. And all stray dust particles down to a very tiny size have to be eliminated from the manufacturing environment in order to prevent defects. Fixing these problems currently seems to be prohibitively expensive.
But even beyond the fairly near-term manufacturing concerns, there are limitations for the devices themselves. One is just that some of the important insulating layers in these circuits are now only a few dozen atoms thick. As these barriers get thinner and thinner, they begin to lose their insulating properties, as electrons quantum-mechanically "tunnel" through them. This will become a problem within about 20 years. Perhaps this particular problem could be alleviated by structuring devices differently or using better insulators such as air. But there are other problems. Semiconductors depend on tiny, trace amounts of impurities mixed in with the Silicon. But when the transistors get very small, these traces only amount to a few atoms in the whole transistor - and tiny random variations in manufacturing will mean a large variation in the transistor's behavior.
Even beyond this, after not too many decades of Moore's law, the entire transistor itself will approach the atomic scale. (See chart.) At this point, thermal noise effects mean that you have to start worrying a lot about keeping the system super-cold in order for it to operate reliably. If the manufacturing capabilities can keep up, eventually we may reach a point where the entire transistor is a single molecule. People are already looking at molecular electronics, and we will talk about that later on in the course. But after you reach the molecular level, then what? We don't have any examples in nature of stable, complex structures any smaller than molecules.
In any event, due to all the manufacturing problems, the semiconductor industry is currently only confidently projecting improvements for another 10-15 years from now. Regardless of precisely what happens, you can rest assured that within your probable lifetime, you will get the chance to witness one of the two startling events: Semiconductor technology will definitely run out of steam as the atomic scale is approached, and either (#1) Moore's law will slow down or stop, which will have drastic consequences for the growth and shape of the economy, since this amazing source of growth of the last half-century will have ended, or (#2) Moore's law will keep on track, but only by shifting over to some radically different kind of technology, such as some form of superconducting, molecular, or quantum computing (all these terms, by the way, start to become roughly synonymous when you get down to the atomic regime). This second possibility would probably also be a really big deal, requiring major upheavals in the manufacturing infrastructure. Either possibility will probably have a big effect on your careers, assuming you will be working in computers - but then, almost every job in the next century will probably involve working with computers to some degree.
One can even look beyond the atomic scale, and postulate that we someday will learn to master even subatomic structures and forces to the extent of being able to use them for computation. Ed Fredkin, former director of the MIT Lab for Computer Science, and one of the early pioneers of computing, has projected from some basic physical limits (which we will cover in this course) that even under this assumption, and even postulating that we were to reprocess the whole solar system into a giant computer, and then spread outwards to the galaxy, expanding our sphere of influence outwards at the speed of light, converting star systems into computers as we go, the total computational power of that whole imagined civilization is still only enough to allow Moore's law to stay on track for another 600 years or so. So it can't go on forever. But perhaps 600 years is long enough.
Philosophically though, it's interesting, because in a way, the whole history of the universe is a history of exponentially-increasing complexity of information processing. After the big bang, the formation of our solar system took 5 or 10 billion years. The first life took another two billion years. Multi-celled life another billion. Hundreds of millions of years to evolve large brains, millions to evolve language and tool-making, tens of thousands of years till agriculture and writing, thousands of years for science and technology, hundreds for printing and industry, tens of years for computing and networking, another few years for the web. It makes one wonder if the next earth-shaking advance might happen in only the next few months! But in any case, if the trend continues, and if Fredkin is right, then we are in the priveliged position of being in the last millenium of the universe's history that will witness such exponential growth of information processing.
Of course, a big caveat to all this is that when we talk about physical limits of computing, we're talking about the limits that follow from physics as it is currently understood. However, physics is not yet complete (see slide) - most egregiously, the basic principles of quantum mechanics and general relativity have not been meshed together at all well yet, and presumably some new conceptual framework will need to arise to reconcile the two views of the world. Potentially when this happens, it could open new opportunities for computation. But at the moment, it seems that the conflicts between general relativity and quantum mechanics only make themselves felt in situations involving extremely high energy densities, such as in black holes, and we are still a very long way from being able to create such extreme conditions (perhaps thankfully!).
So anyway, that sets the stage for this course. We will cover the near-term limits on semiconductor technology. Then, some more generic limits from fundamental physics that will apply to any sort of computing technology. There are some interesting concepts such as reversible computing that come out of this. Then we'll look at some weird alternative (non-semiconductor-based) future computing technologies that various people have proposed.
One of the most interesting is quantum computing, which harnesses some of the counter-intuitive properties of quantum mechanics to compute some problems exponentially faster (as a function of problem size) than any previously-known method. Finally, we'll consider a goal for computer science of characterizing models of computation that mesh well with physics, we'll see what these models might look like, and we'll look at how some of the physical constraints such as reversibility will change of shape of future computer design and programming.
Course overview:
Administrative issues: