CS 327, Spring 2006
Algorithm Analysis and Design for Computational Scientists
HW7

  1. Dividing and conquering a single peak

    A given sequence f(i), where i ranges between 1 and n, is called single-peaked if there is some index k such that f(i) < f(i+1) whenever i < k, and f(i) > f(i+1) whenever i >= k.

    a) Write a program that accepts a single-peaked sequence f(i) as an input, and that finds and prints the maximum value f(k) of the input sequence in time O(log n) by following a divide and conquer strategy. Submit the source code of your program as well as printouts of several sample runs.

    b) Provide a detailed explanation for your claim that your program runs in time O(log n). Your explanation should be based on an analysis of the algorithm on which your program is based, rather than on experimental timing of the running program.

  2. Dynamic programming with loaded coins

    In this task you will develop a dynamic programming approach to computing the probability of observing exactly k heads in n tosses of a "loaded" coin for which the probability of Heads is p. For example, if p=0.7, then the coin will show heads on 70% of the tosses, and tails on the remaining 30% of the tosses, assuming that you try a very long sequence of n tosses. The probability of obtaining some specific sequence of outcomes, say heads, tails, heads (in that order) is the product of the corresponding probabilities, which is 0.7*0.3*0.7 in this example.

    What you aim to compute

    Assume that you have a loaded coin with Heads probability p. For a general value of p, and a very long sequence of n tosses, you'd expect to see heads on about p*n tosses, and tails on the remaining (1-p)*n tosses. However, for shorter sequences of n tosses, there is a significant chance that a different number of heads will be observed. Even a "fair" coin may show an apparent bias towards heads or tails in a typical run of, say, 10 tosses. In general, we'll let prob(n,k) denote the probability of observing exactly k heads in n tosses of the coin. We expect that prob(n,k) will reach a maximum value at about k=np, but prob(n,k) will be non-zero for other values of k as well. We wish to compute these values prob(n,k) in the general case.

    Towards a recurrence relation for this task

    The following argument yields a recurrence relation for prob(n,k) in a manner similar to that for n choose k as discussed in class.

    Suppose that you have already tossed the coin n-1 times. There are two ways of ending up with exactly k heads after the n-th toss.

    case 1). You had k-1 Heads after the (n-1)st toss. The probability of finding yourself in this situation is prob(n-1,k-1). In order to end up with k heads after the n-th toss, you must get Heads on the nth toss. The probability of that happening given that you are in this situation is p.

    case 2). You had k Heads after the (n-1)st toss. The probability of being in this situation is prob(n-1,k). Then you must get Tails on the nth toss in order to end up with exactly k heads. The probability of that happening given that you are in this situation is 1-p.

    Now apply what is called the Chain Rule of Probability in each of the cases 1) and 2). The Chain Rule states that if situation A occurs with probability P(A) and if the probability of a special target event occurring (like ending up with exactly k Heads) given that you are in situation A is x, then the probability of finding yourself in situation A and observing the target event is P(A)*x.

    By the Chain Rule of Probability, the net probability of ending up in case 1) and with exactly k heads after the n-th toss is p*prob(n-1,k-1). Likewise, the net probability of ending up in case 2) and with exactly k heads after the n-th toss is (1-p)*prob(n-1,k).

    Finally, since the target result, ending up with exactly k Heads after n tosses, occurs via either case 1) or case 2) but the two cases cannot occur simultaneously, the Addition Rule of Probability applies: the net probability of observing exactly k Heads in n tosses is the sum of the probabilities obtained in cases 1) and 2).

    In conclusion, the probability of getting exactly k heads after n tosses of the coin is p*prob(n-1,k-1) + (1-p)*prob(n-1,k). Finish writing out the resulting recurrence relation.

    a) Having obtained a recurrence relation for the probability prob(n,k) of getting exactly k Heads in n tosses as described above, sketch the domain of dependence of a given point (n0,k0) in the (n,k) plane for that recurrence relation. Recall that the domain of dependence contains all points (x,y) for which prob(x,y) must be known before prob(n0,k0) can be computed by repeated application of the recurrence relation.

    b) Describe the appropriate initialization for this context, by stating at what points (x,y) in the (n,k) plane the value of prob(x,y) should be determined initially. Note that these must be "boundary points" of the domain of dependence of (n0,k0). Describe also the value of prob(x,y) at these special points.

    c) Using your work in a) and b), write the complete pseudocode for a dynamic programming approach to computing the probability of obtaining exactly k Heads in n tosses of a coin for which P(Heads)=p.

    d) Implement your pseudocode from c) as a program. Submit the source code, as well as sample runs of your program.