Lecture 1, Comp Sc 341s
Overview: What is Complexity?
David Mix Barrington
28 January 2002
- Goals for this initial lecture were as follows:
- Explain the philosophy and overall subject matter of the course,
- Cover the appropriate administrative details, and
- Begin the content of the course with asymptotic notation.
- We propose to study the complexity of computation, meaning
the resources used by computers to solve problems. Our methodology
is that of mathematics rather than engineering. We will define two models
of computation (each with a number of variants): Turing machines and
boolean circuit families. These are abstractions of real computers, but we
will argue that results about them are relevant to the resources used by
computations in the real world.
- You should be somewhat familiar with a Turing machine from a previous
course. It is a digital computer with its memory stored on one or more
tapes -- data structures that store sequences of characters from
some alphabet and can be modified by overwriting the character at a single
location called the head. On a single step, a Turing machine looks
at the character under each tape head, looks at its internal state, and then
decides its new state, which character to write at each head, and which way
to move each head. The Church-Turing Thesis states that any reasonable
model of computation can solve exactly the same class of problems as the
Turing machine, if we allow unlimited resources. We will see that a more
precise version of this thesis is true -- the principal resources used by
the simulating Turing machine are closely related to those used by the
original machine.
- These resources are time, the number of steps taken, and
space, the amount of read/write memory used. We will measure these
resources generally only up to constants, meaning that multiplying
or dividing the time or space by a constant factor will not affect our
results.
- We should also specify what we mean by a problem -- as in your
previous formal language course this will usually mean the decision
problem for a formal language. We fix a finite alphabet, define
a language to be a set of finite strings over that alphabet, and define the
decision problem for a language A to input a string w and output a boolean
saying whether w is in A. A Turing machine does this by starting with the
input on one of its tapes and running until it halts in either an
accepting or rejecting state. We only consider decision algorithms
that halt on any input.
-
Our other model is that of boolean circuit families. A boolean
circuit is a directed graph, where edges (called wires) carry boolean
values and nodes (called gates compute boolean functions. The most
basic gates are AND (binary), OR (binary), and NOT (unary). A circuit must
be acyclic, meaning there can be no directed path from a node to
itself. The source nodes (which I draw at the bottom of a circuit, though
Sipser draws them at the top) represent input values -- individual
bits of an input string in {0,1}*. There is usually one sink
node, representing the boolean output of the circuit (whether w is in some
language A). We will sometimes allow circuits to have more than one output,
in which they compute a function from {0,1}i to {0,1}j
for some numbers i and j.
-
A single circuit has some fixed number of inputs. To solve the decision
problem for a language, we need a family of circuits -- one circuit
Cn for each possible input size n. The resources we will consider
are size, the number of gates in the circuit, and depth, the
length of the longest path from an input to an output. The latter is related
to the parallel time needed to evaluate the circuit. For a particular
family of circuits, size and depth are then functions of the input
size, where size(n) and depth(n) are the size and depth respectively of the
circuit Cn.
- Our basic methodology is as follows. For each model, we define
complexity classes by choosing particular constraints on these
resources and taking the class to be the set of languages whose decision
problems are solvable in that model respecting those constraints. For
example, "P" is the set of languages decidable on a Turing machine in time
that grows polynomially in the input size. We will see many more
examples of complexity classes soon.
-
Our basic results are of two kinds. An upper bound result shows a
problem to be contained in some complexity class, usually by presenting
a solution to the decision problem that obeys the constraints. A lower
bound result shows a problem to be outside some complexity class,
by showing that no possible algorithm that obeys the constraints can solve
the problem. This is naturally much harder than an upper bound, but you have
seen examples in a previous course in the form of proofs that no DFA
can decide the language {anbn: n &ge 0} or that
no Turing machine can decide the halting problem.
-
Finally, an important kind of upper bound result is a simulation.
Here we demonstrate that one complexity class A is contained in another class
B by taking an arbitrary problem in A and showing that it can be solved
in B's model in a way that obeys B's resource constraints. We will show
several relationships between classes defined in terms of Turing machines
and classes defined in terms of circuits. The most interesting classes are
those that are robust, meaning that they can be defined in a variety
of ways.
- That's the background. Administratively, I plan to run this course
recognizing that it is an upper-level elective for a self-selected group of
students. I will expect you to work hard, but largely because you are having
fun rather than because you need this course for a requirement. We have the
freedom to be rather flexible in what we learn.
- Evaluation will be based on a midterm (20%), final exam (30%), homework
(30%) assigned weekly, and a project (20%) to be presented both orally and
in writing at the end of the term. I have no preconceived notion of how
many students will get which grades -- I hope you all deserve A's and if you
do you will get them. Collaboration on the homework is encouraged but the
presentation of the work you hand in must be your own individual work.
- The textbook is Introduction to the Theory of Computation by
my Ph.D. advisor, Michael Sipser. We will cover most of chapters 7-10 of
this book but many of the topics will be presented in a different order.
Another good source of material is the set of lecture notes from the 2000
Park City Math Institute summer program, available at
this web site, maintained
by my colleague Alexis Maciel. These address topics in the order I will
use in this course -- we will probably cover Basic Lectures 1-10.
- The prerequisite for this course is COMP SC 311, which covers around
the first five chapters of Sipser. The more math you know and the more you
like math, the better you will like this course, but there is no specific
mathematics prerequisite -- I will work with whatever you know.
- We will need the language and notation of asymptotic bounds on
functions to state all of our resource constraints. If f and g are functions
from the non-negative integers to the non-negative integers, we define the
statement "f = O(g)" to be true if the following holds. There must be some
constant number c such that f(n) &le cg(n) for all sufficiently large
numbers n. (This means that there exists a number n0 such that
f(n) &le cg(n) holds for all n greater than n0.)
- We use this notation to say, for example, that a particular algorithm
uses "time O(n)" to solve a problem. That is, the function time(n), giving
the worst-case running time on inputs of length n, satisfies
time(n) = O(n) and thus for some c and sufficiently large n, time(n) &le
cf(n). This notation has a number of advantages. Changes in our model that
multiply or divide the running time by a constant, such as a change in the
"instruction set" allowing one new step to do the work of several old steps
or vice versa, do not affect the question of whether time(n) = O(n).
- We will say more about asymptotic bounds in the next lecture, and
discuss the particular bounds that we will use to define complexity classes.
Last modified 1 February 2002
David A Mix Barrington