The Knapsack Problem

The knapsack problem is a classic CS problem. This is the text:

A thief robbing a safe finds it filled with $N$ items. You want to steal the most monetary value while it all fits in your knapsack with a constant capacity.

Assume that this knapsack has capacity $M$ and $N$ items in the safe.

There are several variations:

• Each item is unique (the 0-1 knapsack problem)
• Finite amount of each item (explicit list)
• Infinite amount of each item
• Fractional amount of each item
• Using weight ($w_i$) instead of size

There are tons of possible solutions, but we're going to look at three:

• Brute force
• Greedy
• Dynamic programming

Brute Force

First, generate all possible solution space. Consider all possible subsets of the $N$ items. Filter these out to the feasible solutions, and then find the one with the highest value.

bool array possSet[1..N]
int maxVal = 0
for int i = 1 to 2^N
    possSet[] = genNextPOwer(N)
    int setSize = findSize(possSet[])
    int setValue = findValue(possSet[])
    if setSize <= M and setValue > maxVal
        bestSet[] = possSet[]
        maxVal = setValue
return maxVal

Efficiency

• You had to generate all possible sets: $O(2^n)$
• Filter feasible solution set: $O(n)$
• Determine optimal solution: $O(n)$

In total, this has a complexity of $O(N*2^N)$. This is pretty hideous.

Greedy

There are several approaches:

• Steal the highest value items first
• Steal the lowest weight items first
• Steal the highest value to weight ratio items first

maxVal = 0, currentSize = 0
ratio[] = buildRatio(value[], size[])
sortedRatio[] = sortRatio(ratio[])

for int i = 1 to N
    if size[i] + currSize <= M
        currSize = currSize + size[i]
        maxVal = maxVal + value[i]

return maxVal

Efficiency

Most of the time here is spent sorting.

• Sorting: $O(N \log N)$
• Choose item with highest ratio first: $O(N)$

In total, this is $O(N \log N)$. Not bad! However, this does not always give an optimal solution.

Now, what happens when you do a fractional problem?

Fractional Knapsack: Greedy

If you have a fractional greedy ratio, this does yield a perfect solution. You can just take exactly as much as you want for each item.

Dynamic Programming

We're going to look at three approaches with the goal of getting the right answer fast:

• Simple recursive (non-DP)
• Top-down: recursive DP
• Bottom-up: iterative DP

Note that we are assuming an infinite amount of each item.

Recursive Approach

For each item, place it in the knapsack, find the optimal packing for a smaller knapsack, and remember the best packing. The algorithm is a direct recursive solution, but takes exponential time.

Algorithm knapsack(int capacity)
    max_val = 0
    for each item in N
        space_rem = capacity - item.size
        if (space_rem >= 0)
            new_val = knapsack(space_rem) + item.val
            if (new_val > max_val) max_val = new_val
    return max_val

This is exponential time. Not as bad as brute force, but NP.

There are many overlapping sub-problems here. We call the knapsack function many times for repeated values of space_rem. Let's make this better.

Algorithm knapsack(int capacity)

    // CHECK THIS OUT
    if (maxKnown[capacity] is known)
        return maxKnown[capacity]

    max_val = 0
    for each item in N
        space_rem = capacity - item.size
        if (space_rem >= 0)
            new_val = knapsack(space_rem) + item.val
            if (new_val > max_val) max_val = new_val

    // CHECK THIS OUT TOO
    maxKnown[capacity] = max_val

    return max_val

This is $O(MN)$.