Lecture 16 for Math 232

Introduction to Counting Problems

1 April 2002

• Today's lecture had four main goals:
  1. Discuss the general setting of counting problems.
  2. Discuss the sum rule for counting unions of disjoint sets.
  3. Discuss the product rule for counting direct products of sets.
  4. Define bit strings and discuss how to count sets of bit strings with certain characteristics.
  5. Discuss the double-counting rule, to count thet union of two sets even if they are not disjoint.

  This material is in section 4.1 of Rosen.

• The field of combinatorics studies how to determine the number of objects in some finite set. It has many applications in computer science and mathematics. For example, we may want to know how many possible addresses in a computer can be specified in a certain format. Or we may want to treat the terms in a large algebraic expression in an organized way, which usually begins with determining how many there are.

• When we count a set in real life, we must make sure of two things:
  • Every object is counted, and
  • No object is counted twice.

• In effect, we have defined a function from the numbers {1,...,n} to our set, where f(i) is the object we pointed to when we counted "i". This function must be onto if every object is counted, and it must be one-to-one if no object is counted twice. A function that is both one-to-one and onto is called a bijection. The most fundamental fact in combinatorics is that if a bijection exists from some set A to some set B, then A and B have the same number of elements. Typically we count some new set A by defining a bijection from A to some set whose size we know -- this is called a combinatorial proof that A's size is the same as B's.

• Suppose A and B are two sets with no elements in common (they are disjoint. Then if we first count A using the numbers 1 through a, and then count B using the numbers a+1 through a+b, we find that we have counted the elements of (A UNION B) correctly. We counted every element once, and no element more than once because no element was in both A and B. We have proved the Sum Rule, that if A and B are disjoint, then the size of (A UNION B) is the size of A plus the size of B. Rosen states this rule slightly differently: "if we have two disjoint sets of choices, the number of ways to choose from A or choose from B is the size of A plus the size of B". This means the same thing but may sometimes be easier to apply.

• Now consider the direct product (A TIMES B) of two sets A and B. This consists of all pairs (a,b), where a is any element of A and b is any element of B. The Product Rule says that if A has size m and B has size m, then (A TIMES B) has size mn. It is important that any pair in (A TIMES B) be allowable for the number to be mn.

• We can prove the Product Rule using the Sum Rule and mathematical induction. Let P(n) be the statement that (A TIMES B) has size mn whenever A has size m (which we treat as arbitrary) and B has size n. The base case P(0) says that if B is the empty set, (A TIMES B) has size 0. This is true because (A TIMES B) is itself empty -- there can be no pair whose second element is from B because there are no elements in B. Now for the induction case. Assume that P(n) is true -- that for any A of size m and any B of size n, (A TIMES B) has size mn. We set out to prove P(n+1). Let C be an arbitrary set of size n+1. Let x be any element of C, and let B = C - {x}. Since B has n elements, (A TIMES B) has mn elements by the IH. But (A TIMES C) is the union of (A TIMES B) and (A TIMES {x}), because a pair in (A TIMES C) is either (a,b) where b is in B or is (a,x). Furthermore, these two sets are disjoint, and (A TIMES {x}) has size m because it has a bijection with A (taking a to (a,x)). Thus the size of (A TIMES C) is mn + m or m(n+1), and the statement P(n+1) is proved (since C was arbitrary). This completes the induction.

• We can use the Product Rule repeatedly to count the number of bit strings of any given length n. A bit string is a sequence of 0's and 1's, and its length is the number of letters in it. We can prove by induction that the number of bit strings of length n is 2ⁿ. P(0) says that the number of bit strings of length 0 is 1, and this is true because there is exactly one empty string. We assume P(n), and set out to prove P(n+1). The set of bit strings of length n+1 is in a bijection with the direct product of the bit strings of length n with the set {0,1}, because for any string w we can map the pair (w,0) or (w,1) to the string w0 or w1. By the product rule, then the number of strings of length n+1 is 2ⁿ times 2 or 2ⁿ⁺¹, where the 2ⁿ is from the IH.

• We can count a partially specified set of bit strings by using the product rule more carefully. For example. consider the set of bit strings of length n that start with 0 and end with 1. If n < 2 there are no such strings, but for n &ge 2 we may form such a string by choosing a bit for each position other than the first and last. Since this is n-2 choices, our number is 2^n-2.

• What about the set of strings of length n that either begin with 0 or end with 1? This is the union of the strings beginning with 0 and the strings ending with 1, but as these two sets are not disjoint in general we cannot use the sum rule directly (unless n < 2). Each of these two sets has size 2^n-1, but if we just add the sizes we get 2ⁿ which is the number of all the strings of length n, clearly too many as some strings do not meet either requirement. We have double-counted the strings that both begin with 0 and end with 1.

• The Double-Counting Rule allows us to correct this problem. It says that the size of (A UNION B) is the size of A, plus the size of B, minus the size of (A INTERSECT B). For disjoint sets, this simply becomes the sum rule. In our example, the sizes of A and B are each 2^n-1 and the size of (A INTERSECT B) is 2^n-2, giving us the correct answer of 2ⁿ - 2^n-2 or (3/4)2ⁿ.

• Why is the double-counting rule true? We can prove it using the sum rule. (A UNION B) is the union of three sets, (A - B), (B - A), and (A INTERSECT B). Furthermore, these three sets are pairwise disjoint, in that no element is in more than one of them. If we let x, y, and z respectively be the sizes of these three sets, we get that the size of (A UNION B) is x+y+z by the sum rule, and that the sizes of A and B are x+z and y+z respectively. The double-counting rule follows by arithmetic: (x+y+z) = (x+z) + (y+z) - z.

• Rosen calls the double-counting rule the "Rule of Inclusion/Exclusion", but I prefer to reserve that name for the more complicated rule that deals with the union of three or more sets. For example, letting "|X|" denote the size of X, we can use the sum rule to prove that:

  |A UNION B UNION C| = |A| + |B| + |C| - |A INTERSECT B| - |A INTERSECT C| - |B INTERSECT C| + |A INTERSECT B INTERSECT C|.

  We will see this rule later (when we do section 5.5 of Rosen).

• One final note: we have stated and proved the sum and product rules only for unions of two disjoint sets, or products of two sets. Each rule has a more general form:
  • General Sum Rule: If A₁,...,A_n are pairwise disjoint sets, then |A₁ UNION ... UNION A_n| = |A₁| + ... + |A_n|.
  • General Product Rule: If A₁,...,A_n are any sets, |A₁ TIMES ... TIMES A_n| = |A₁| * ... * |A_n|.

• These two general rules are not hard to prove by induction from the two-set rules -- this is a good exercise for you to try.

Last modified 26 April 2002