How to use Solver & Dynamic Programming to solve this problem?

Since I don't work in computer science (at least, not to that level), I figure a lot of this question is really about "what is dynamic programming." So, in the spirit of "let me google that for you" I put "dynamic programming into my favorite search engine and found a couple of different wikipedia pages as well as several tutorials/introductions to the idea of dynamic programming. Perhaps one of these will remind you how this was discussed in class:
http://en.wikipedia.org/wiki/Dynamic_programming
http://20bits.com/article/introducti...ic-programming
http://www.cprogramming.com/tutorial...theory/dp.html

In short, it would appear that "dynamic programming" is simply a reference to what kind of algorithm you might come up with to ultimately optimize this kind of problem. I'm not sure if they are expecting you to program some kind of VBA/spreadsheet "what-if" scenario for this, or if they are saying that Solver has the dynamic programming algorithms already built in, or exactly what they are intending for you. I wish I could help more.