Dynamic Programming (DP) was developed by Richard Bellman in the 1950s and has found applications in numerous fields, from aerospace engineering to economics.
DP is an algorithm technique that is closely related to the divide and conquer approach. However, while the divide and conquer approach is essentially recursive, and so "top down", dynamic programming works "bottom up".
This method is an optimisation approach that transforms a complex problem into a sequence of simpler problems. It creates an array of related but simpler sub-problems, and then, it computes the solution to the big complicated problem by using the solution to the easier sub-problems which are stored in the array. In this way, the efficiency of the CPU can be enhanced.
DP solutions have a polynomial complexity which assures a much faster running time than other techniques like backtracking, brute-force etc. For example, the following simple recursive solution (top down) for Fibonacci Numbers gets exponential time complexity.
int fib (int n) {
if (n <= 1)
return n;
return fib(n - 1) + fib(n - 2);
}
The time complexity reduces to linear in the below-optimised code (bottom up).
std::vector<int> f{0, 1};
for (int i = 2; i <= n; ++i) {
f.emplace_back(f[i - 1] + f[i - 2]);
}
return f[n];
DP works by storing the result of sub-problems so that when their solutions are required, they are at hand and we do not need to recalculate them. The technique of storing the results of already solved sub-problems is called memoisation. By saving the values in the array, we save time for computations of sub-problems we have already come across.
The Manacher's algorithm is the problem of finding the longest substring which is a palindrome.
The time complexity of the algorithm is O(n).
The basic idea of the algorithm is to find the symmetric string on both sides of the centre. For example, abcdcba and madamisimadam. If the length of the string is even, a special character should be added as follows:
S: abba
S´: |a|b|b|a|
The length of the right string is 9, 2 * length(S) + 1, which is odd.
Let P be an array of S´:
|
a
|
b
|
b
|
a
|
Each position of the array can be the centre to calculate the length of the palindromic string of the positions.
Depending on the position of the centre, the palindromic lengths are stored as below:
Index
0
1
2
3
4
5
6
7
8
S´
|
a
|
b
|
b
|
a
|
Len
0
3
0
3
5
3
0
3
0
In the array, the centre of the longest palindromic string is at index 4.
Pseudocode
Longest_Palindrome(string S) {
string S' = S with a bogus character (eg. '|') inserted between each character (including outer boundaries)
array PalindromeRadii = [0,...,0]
// The radius of the longest palindrome centered on each place in S'
// note: length(S') = length(PalindromeRadii) = 2 × length(S) + 1
Center = 0
Radius = 0
while Center < length(S')
{
// At the start of the loop, Radius is already set to a lower-bound for the longest radius.
// In the first iteration, Radius is 0, but it can be higher.
// Determine the longest palindrome starting at Center-Radius and going to Center+Radius
while Center-(Radius+1) >= 0 and Center+(Radius+1) < length(S') and S'[Center-(Radius+1)] = S'[Center+(Radius+1)] {
Radius = Radius+1
}
// Save the radius of the longest palindrome in the array
PalindromeRadii[Center] = Radius
// Below, Center is incremented.
// If any precomputed values can be reused, they are.
// Also, Radius may be set to a value greater than 0
OldCenter = Center
OldRadius = Radius
Center = Center+1
// Radius' default value will be 0, if we reach the end of the following loop.
Radius = 0
while Center <= OldCenter + OldRadius {
// Because Center lies inside the old palindrome and every character inside
// a palindrome has a "mirrored" character reflected across its center, we
// can use the data that was precomputed for the Center's mirrored point.
MirroredCenter = OldCenter - (Center - OldCenter)
MaxMirroredRadius = OldCenter + OldRadius - Center
if PalindromeRadii[MirroredCenter] < MaxMirroredRadius {
PalindromeRadii[Center] = PalindromeRadii[MirroredCenter]
Center = Center+1
}
else if PalindromeRadii[MirroredCenter] > MaxMirroredRadius {
PalindromeRadii[Center] = MaxMirroredRadius
Center = Center+1
}
else { // PalindromeRadii[MirroredCenter] = MaxMirroredRadius
Radius = MaxMirroredRadius
break // exit while loop early
}
}
}
longest_palindrome_in_S' = 2*max(PalindromeRadii)+1
longest_palindrome_in_S = (longest_palindrome_in_S'-1)/2
return longest_palindrome_in_S
}
The window sliding algorithm is used to convert two nested loops (bruth force) into a single loop (linear), which means it enables us to reduce the time complexity from O(n²) to O(n).
The following is an example of converting the bruth force approach into the window sliding technique.
Given an array A = { 1, 3, 5, 6, 8, 10 }, find the maximum sum of 3 consecutive elements.
Bruth Force
int maxSum = 0;
for (int i = 0; i < A.size() - 3; i++)
{
int sum = 0;
for (int j = 0; j < 3 + 1; j++)
{
sum += A[i + j];
}
maxSum = max(sum, maxSum);
}
return maxSum;
WindowSliding
int maxSum = 0;
int panes = 0;
int windowSum = 0;
for (int i = 0; i < sizeA + 1; i++)
{
if (panes == 3)
{
maxSum = max(maxSum, windowSum);
windowSum -= A[i - 3];
panes--;
}
windowSum += A[i];
panes++;
}
Hopcroft-Karp algorithm (sometimes more accurately called the Hopcroft-Karp-Karzanov algorithm) is an algorithm for solving the maximum matching problem in a bipartite graph. This algorithm produces a maximum cardinality matching as output and runs in O(E√V) time in the worst case, where E is set of edges in the graph and V is set of vertices of the graph.
Bi-partite graph (also known as bigraph)
A graph G(V,E) is called a bipartite graph. V and E are usually called the parts of the graph.
If the set of vertices V can be partitioned in two no-empty disjoint sets V1 and V2 in such a way that each edge e in G has one endpoint in V1 and another endpoint in V2.
Complete bipartite graph
If each vertex in A is adjacent to all the vertices in B, then the graph is a complete bipartite graph and gets a special name: Km,n, where |A| = m and |B| = n.
Matching
A set of pairwise non-adjacent edges, none of which are loops; that is, no two edges share a common endpoint.
A vertex is matched (or saturated) if it is an endpoint of one of the edges in the matching.
Otherwise, the vertex is unmatched.
A matching in a bipartite graph is a set of the edges chosen in such a way that no two edges share an endpoint.
Maximum matching (also known as Maximum-Cardinality Matching)
A maximum matching is a matching of maximum size (maximum number of edges).
If any edge is added, it is no longer matching.
For example, a maximum of five people can get jobs in the following graph of an assignment of jobs to applicants.
The blue-coloured path is matched while the black one is unmatched.
Hopcroft-Karp algorithm is often outperformed by Breadth-First and Depth-First approaches to finding augmenting paths.
Augmenting paths
Given a matching M, an augmenting path is an alternating path that starts from and ends on free vertices.
2 and b are free vertices.
A walk from 2 to c and a walk from 4 to b are augmented paths.
How Hopcroft-Karp algorithm works
Define two sets of vertices from the bipartition of G, U and V, with an equal number of nodes.
The matchings, or set of edges, between nodes in U and nodes in V, is called M.
The algorithm runs in phases mode up of the following steps:
Use a BFS to find augmenting paths. If partitions the vertices of the graph into layers of matching and not matching U. For the search, start with the free nodes in U. This forms the first layer of the partitioning. The search finishes at the first layer k where one or more free nodes in V are reached.
The free nodes in V are added to a set called F. This means that any node added to F will be the ending node of an augmenting pathㅡand a shortest augmenting path at that since the BFS finds the shortest paths.
Once an augmenting path is found, a DFS is used to add augmenting paths to current matching M. At any given layer, the DFS will follow edges that lead to an unused node from the previous layer. Paths in the DFS tree must be alternating paths (switching between matched and unmatched edges). Once the algorithm finds an augmenting path that uses a node from F, the DFS moves on to the next starting vertex.
Observe the graph with edges but no assigned matchings.
Step 1: Perform BFS starting at all the numeric vertices without a match. Pick any unmatched leaf and go all the way back to a root using DFS. Match the leaf to the root.
Match a to 1.
Delete all the instances of 1 and a found in the trees.
Step 2: Repeat the process to find the next matching.
Match b to 2.
Delete b from the tree spanning from 3.
In this iteration, 1 is matched to a, 2 is matched to b, and 3, along with c, is left without a match.
Step 3: Since 3 is left without a match, perform BFS starting from 3 in order to find a matching or assert that the current matching is already optimal.
Unmatched edges!
Perform DFS once again from the unmatched leaf all the way to the root.
DFS finds a path from c to 1, to 1, and terminates at 3.
Step 4: Augment the path by switching the edges that are matched with those that are unmatched.
Maximal matching and termination of the algorithm
New 1 is matched with c and 3 is matched with a.
This produces the maximal matching between numbers and letters in G.
