Lecture 1, Comp Sc 341s

Overview: What is Complexity?

David Mix Barrington

28 January 2002

• Goals for this initial lecture were as follows:
  1. Explain the philosophy and overall subject matter of the course,
  2. Cover the appropriate administrative details, and
  3. Begin the content of the course with asymptotic notation.

• We propose to study the complexity of computation, meaning the resources used by computers to solve problems. Our methodology is that of mathematics rather than engineering. We will define two models of computation (each with a number of variants): Turing machines and boolean circuit families. These are abstractions of real computers, but we will argue that results about them are relevant to the resources used by computations in the real world.

• You should be somewhat familiar with a Turing machine from a previous course. It is a digital computer with its memory stored on one or more tapes -- data structures that store sequences of characters from some alphabet and can be modified by overwriting the character at a single location called the head. On a single step, a Turing machine looks at the character under each tape head, looks at its internal state, and then decides its new state, which character to write at each head, and which way to move each head. The Church-Turing Thesis states that any reasonable model of computation can solve exactly the same class of problems as the Turing machine, if we allow unlimited resources. We will see that a more precise version of this thesis is true -- the principal resources used by the simulating Turing machine are closely related to those used by the original machine.

• These resources are time, the number of steps taken, and space, the amount of read/write memory used. We will measure these resources generally only up to constants, meaning that multiplying or dividing the time or space by a constant factor will not affect our results.

• We should also specify what we mean by a problem -- as in your previous formal language course this will usually mean the decision problem for a formal language. We fix a finite alphabet, define a language to be a set of finite strings over that alphabet, and define the decision problem for a language A to input a string w and output a boolean saying whether w is in A. A Turing machine does this by starting with the input on one of its tapes and running until it halts in either an accepting or rejecting state. We only consider decision algorithms that halt on any input.

• Our other model is that of boolean circuit families. A boolean circuit is a directed graph, where edges (called wires) carry boolean values and nodes (called gates compute boolean functions. The most basic gates are AND (binary), OR (binary), and NOT (unary). A circuit must be acyclic, meaning there can be no directed path from a node to itself. The source nodes (which I draw at the bottom of a circuit, though Sipser draws them at the top) represent input values -- individual bits of an input string in {0,1}^*. There is usually one sink node, representing the boolean output of the circuit (whether w is in some language A). We will sometimes allow circuits to have more than one output, in which they compute a function from {0,1}ⁱ to {0,1}^j for some numbers i and j.

• A single circuit has some fixed number of inputs. To solve the decision problem for a language, we need a family of circuits -- one circuit C_n for each possible input size n. The resources we will consider are size, the number of gates in the circuit, and depth, the length of the longest path from an input to an output. The latter is related to the parallel time needed to evaluate the circuit. For a particular family of circuits, size and depth are then functions of the input size, where size(n) and depth(n) are the size and depth respectively of the circuit C_n.

• Our basic methodology is as follows. For each model, we define complexity classes by choosing particular constraints on these resources and taking the class to be the set of languages whose decision problems are solvable in that model respecting those constraints. For example, "P" is the set of languages decidable on a Turing machine in time that grows polynomially in the input size. We will see many more examples of complexity classes soon.

• Our basic results are of two kinds. An upper bound result shows a problem to be contained in some complexity class, usually by presenting a solution to the decision problem that obeys the constraints. A lower bound result shows a problem to be outside some complexity class, by showing that no possible algorithm that obeys the constraints can solve the problem. This is naturally much harder than an upper bound, but you have seen examples in a previous course in the form of proofs that no DFA can decide the language {aⁿbⁿ: n &ge 0} or that no Turing machine can decide the halting problem.

• Finally, an important kind of upper bound result is a simulation. Here we demonstrate that one complexity class A is contained in another class B by taking an arbitrary problem in A and showing that it can be solved in B's model in a way that obeys B's resource constraints. We will show several relationships between classes defined in terms of Turing machines and classes defined in terms of circuits. The most interesting classes are those that are robust, meaning that they can be defined in a variety of ways.

• That's the background. Administratively, I plan to run this course recognizing that it is an upper-level elective for a self-selected group of students. I will expect you to work hard, but largely because you are having fun rather than because you need this course for a requirement. We have the freedom to be rather flexible in what we learn.

• Evaluation will be based on a midterm (20%), final exam (30%), homework (30%) assigned weekly, and a project (20%) to be presented both orally and in writing at the end of the term. I have no preconceived notion of how many students will get which grades -- I hope you all deserve A's and if you do you will get them. Collaboration on the homework is encouraged but the presentation of the work you hand in must be your own individual work.

• The textbook is Introduction to the Theory of Computation by my Ph.D. advisor, Michael Sipser. We will cover most of chapters 7-10 of this book but many of the topics will be presented in a different order. Another good source of material is the set of lecture notes from the 2000 Park City Math Institute summer program, available at this web site, maintained by my colleague Alexis Maciel. These address topics in the order I will use in this course -- we will probably cover Basic Lectures 1-10.

• The prerequisite for this course is COMP SC 311, which covers around the first five chapters of Sipser. The more math you know and the more you like math, the better you will like this course, but there is no specific mathematics prerequisite -- I will work with whatever you know.

• We will need the language and notation of asymptotic bounds on functions to state all of our resource constraints. If f and g are functions from the non-negative integers to the non-negative integers, we define the statement "f = O(g)" to be true if the following holds. There must be some constant number c such that f(n) &le cg(n) for all sufficiently large numbers n. (This means that there exists a number n₀ such that f(n) &le cg(n) holds for all n greater than n₀.)

• We use this notation to say, for example, that a particular algorithm uses "time O(n)" to solve a problem. That is, the function time(n), giving the worst-case running time on inputs of length n, satisfies time(n) = O(n) and thus for some c and sufficiently large n, time(n) &le cf(n). This notation has a number of advantages. Changes in our model that multiply or divide the running time by a constant, such as a change in the "instruction set" allowing one new step to do the work of several old steps or vice versa, do not affect the question of whether time(n) = O(n).

• We will say more about asymptotic bounds in the next lecture, and discuss the particular bounds that we will use to define complexity classes.

Last modified 1 February 2002