Algorithm Families

Brute Force

A brute force algorithm doesn't follow any design patterns. It often takes the form of the most simple solution to implement, but might not be very efficient.

Pros:

• Simplicity

Cons:

• May do more work than necessary
• May be inefficient (but typically it's not)
• Sometimes, not that obvious

Greedy Algorithms

A greedy algorithm is something that makes a sequence of decisions, and never reconsiders decisions that have been made already.

You must show that the locally optimal decision leads to a globally optimal solution.

Pros:

• May run significantly faster than the brute-force solution

Cons:

• May not lead to the correct/optimal solution.

Example: Sorting

Let's say you're looking at sorting. A brute force solution: you could create all the $n!$ permutations, and see which one is sorted.

Another approach to sorting leads to a better solution. You could find the smallest item, move to the first location ($n$ operations), and then find the next smallest item and move it to the second location ($n - 1$ operations), and leave the largest item in the final location. This is selection sort.

Bubble sort is an example of a greedy approach to sorting.

Divide and Conquer

Divide a problem solution into two (or more) smaller problems, preferably of equal size. This is often recursive, and often involve a $\log n$ factor. Why? Dividing it brings an expectation of a log somewhere in the results.

Pros:

• Efficiency
• 'Elegance' of recursion

Cons:

• Recursive calls to small subdomains often expensive
• Sometimes is dependent upon the initial state of the subdomains
  • Example: binary search requires sorted array

Combine and Conquer

Let's start with a smallest subdomain possible. Then, combine increasingly larger subdomains until size = n. An example of this is merge sort.

Dynamic Programming

Remembers partial solutions when smaller instances are related. It solves the small instances first, stores the results, and looks them up when needed.

Pros:

• Can make 'brutally' inefficent algorithm very efficient (sometimes $O(2^n)$ to $O(n^c)$)!

Cons:

• Difficult algorithmic approach to grasp

Types of Algorithm Problems

Constraint Satisfaction problems:

• Can we satisfy all given constraints?
• If yes, how do we satisfy them?
• May have more than one solution
• Examples: sorting, mazes, spanning tree

With these problems, you could stop early.

Optimization problem: Constraint problem with objective function to maximize/minimize

• Giving change, MST
• Can't always stop early
• Must develop set of possible solutions, then pick best

Backtracking Algorithms

Definition: systematically consider all possible outcomes of each decision, but prune searches that do not satisfy the constraints

• Think of DFS with pruning
• Eliminates the exhaustive search

Graph Properties

• Maps can be drawn with planar graphs: a graph that can be drawn with no crossing edges

Can a graph be colored with four colors? To do this, assign colors to vertices such that no two vertices connected by an edge have the same color. Some graphs can be 4-colored, and some cannot.

Start with some vertex on a graph of 5 vertices. It has to be colored. Suppose you pick vertex A, then B, then C, then D, etc. Let's say you color A the color 1. You never have to consider the graph where vertex A is colored anything else. The choice of one color does not matter. You've just reduced the search space from $4^5$ to 4^4$$. A little bit better.