Graph Traversal: BFS and DFS

Recall that we have the following tree traversals:

• BFS-based (non-recursive/queue):
  • level-order
• DFS-based (recursive/stack):
  • pre-order
  • in-order
  • post-order

Similarly, we can traverse a graph using BFS and DFS. Just like in trees, both BFS and DFS color each node in three colors in the traversal process:

• white: not visited at all; not in queue/stack yet
• gray: active nodes; being visited but not done yet; currently in queue/stack
• black: complelety explored; popped out of queue/stack.

BFS

BFS for graph is exactly like BFS for tree, except that now you need to check whether a node has been visited before when you try to add it to the queue. You only want to add a “white” node that has never been seen before to the queue.

def bfs(n, edges): # full BFS procedure for the whole graph
    def BFS(v): # do one BFS from v
        queue = [v] # start node
        for u in queue: # pop u logically but not physically from queue
            order.append(u) # BFS visit order 
            color[u] = 2 # black    
            for w in adjlist[u]: # u->w
                if color[w] == 0: # only if white
                    queue.append(w)
                    color[w] = 1 # gray

    adjlist = edges2adjlist(edges) # convert edges to adjaency list
    color = defaultdict(int) # default 0: white
    order = [] # output
    for v in range(n): # no need for this loop if there is "God" node
        if color[v] == 0: # only do BFS on white nodes
            BFS(v) # another BFS tree in BFS forest
    return order

Caveats:

• Note we need the outmost for-loop because the graph might be disconnected or weakly connected, so calling BFS(v) may only reach part of the graph reachable from v (and produces a BFS tree rooted at v). In order to make sure every node is visited, we need this extra for loop (the set of BFS trees is called the BFS forest, see below). Alternatively, we can add a hidden “God” node which connects to every real node, and in that case, we don’t need this extra for-loop because we just need to call BFS(god_node).
  
• What’s the easiest way to maintain the queue? There are three implementations:
  • The most obvious and naive way is to use a linked list where you can do both pop-left and push-right in \(O(1)\) time. In Python, you can use the deque class which is double-ended queue (so that you can pop/push on both ends in \(O(1)\) time), implemented as a doubly-linked list.
  • But deque is an overkill since we just need a queue. Can you use something simpler, e.g., an array? But while array supports push-right (append) in \(O(1)\) time, its pop-left is \(O(n)\). How to solve this dilemma? Well, actually you don’t need to remove the head in a pop-left operation: just leave it there in the array but maintain a head pointer. Now pop-left becomes head += 1, which is a “logical” but not “physical” pop. The for loop to traverse the elements in the (dynamic) queue becomes while head < len(queue): and head == len(queue) means the queue is empty. This is the standard implementation found in most textbooks.
  • However, in Python, there is a third way based on the second but even simpler: you don’t need to explicitly maintain the head pointer, just do for u in queue: and it works even if queue is expanding on the right end! This the recommended solution.

For example,

0 --> 6 --> 2 --> 3 <-- 7 --> 8 
  /   |           v 
1     +---> 4 --> 5

>>> print(bfs(9, [(0,6), (1,6), (6,2), (2,3), (6,4), (4,5), (3,5), (7,3), (7,8)]))

[0, 6, 2, 4, 3, 5, 1, 7, 8]

Note that there are three BFS trees in this BFS forest:

BFS(0)  BFS(1)  BFS(7)
    0       1       7
    |               |
    6               8
   / \
  2   4
  |   |
  3   5

The time complexity for BFS is \(O(V+E)\) (linear in the size of the graph) because you need to visit each edge once and only once, and each node is added to the queue once and popped from the queue once.

Application of BFS in Shortest-Path

BFS can be used to find the single-source shortest-path(s) in unweighted graphs (where each edge has a unit cost), which is also known as “uniform cost” search in AI. Basically, the first time you add a node \(v\) to the queue, it is guaranteed that \(v\)’s optimal distance for the source has been found. Therefore, in single-source-single-destination problems, when the target node \(t\) is added to the queue, you can terminate immediately. BFS is indeed the fastest algorithm for shortest-path on unweighted graphs. For example, for the coins problem (minimum number of coins to make up an amount), BFS would be much faster than Viterbi (or DP).

By contrast, in a general weighted graph, nodes in the queue can still be updated to better distances, and therefore you need to use a priority queue instead of a queue, and this becomes the Dijkstra algorithm. So Dijkstra is a generalization of BFS on weighted graphs.

DFS

DFS becomes more interesting on graphs, especially directed graphs. Unlike BFS, here we use a stack, and like BFS, we only push a node to the stack if it’s white. In real implementation, like in trees, we use recursion which maintains the implicit stack for you. The following DFS code also detects cycles as a by-product:

def dfs(n, edges): # DFS for the whole graph
    def DFS(v): # recursive
        color[v] = 1 # gray
        order.append(v)
        for u in adjlist[v]: # v->u
            if color[u] == 0: # tree edge
                DFS(u)
            elif color[u] == 1: # gray: active; back edge
                print("cycle detected %d->%d" % (v, u)) # this part is optional
        color[v] = 2 # black

    adjlist = edges2adjlist(edges) # convert edges to adjaency list
    color = defaultdict(int) # default 0: white
    order = [] # output
    for v in range(n): # no need for this loop if there is "God" node
        if color[v] == 0:
            DFS(v) # another DFS tree

    return order

Note that

• because we don’t need to maintain an explicit stack, this DFS code is a bit shorter than BFS where we need to maintain a (logical) queue;
• those on the implicit stack (active nodes) are exactly the set of gray nodes; so even for cycle-detection (yes or no) there is no need to maintain the stack.
• again, if there is a God-node then the for-loop in the main function is not needed.

For example,

0 --> 6 --> 2 --> 3 <-- 7 --> 8 
  /   |           v 
1     +---> 4 --> 5

>>> print(dfs(9, [(0,6), (1,6), (6,2), (2,3), (6,4), (4,5), (3,5), (7,3), (7,8)]))
[0, 6, 2, 3, 5, 4, 1, 7, 8] # different from BFS order

Here is the DFS forest; as you can see DFS trees (being “depth”-first) are indeed “deeper” than BFS ones:

DFS(0)  DFS(1)  DFS(7)
    0       1       7
    |               |
    6               8
   / \
  2   4
  |   
  3
  |
  5

Now if we add to edges, 5->1 and 5->7 to form cycles:

0 --> 6 --> 2 --> 3 <-- 7 --> 8 
  /   |           v     ^
1     +---> 4 --> 5 ----+
^                 |
  \---------------+

>>> print(dfs(9, [(0,6), (1,6), (6,2), (2,3), (6,4), (4,5), (3,5), (7,3), (7,8), (5,1), (5,7)]))
cycle detected 1->6
cycle detected 7->3
[0, 6, 2, 3, 5, 1, 7, 8, 4]

It detects cycle twice, producing this DFS forest with a single, deep, tree (with back edges shown in dotted edges):

  DFS(0)
      0 
      | 
....> 6 
.    / \
.   2   4
.   |   
.   3 <..
.   |   .
.   5   .
.  / \  .
..1   7..
      |
      8  

Our code can only detect the existence of cycles. If you also want to print a cycle (6->2->3->5->1->6 and 3->5->7) then you do need to maintain the stack. Note there are many other “related” cycles that are not detected in DFS, such as 6->4->5->1->6.

Like BFS, the time complexity for DFS is also \(O(V+E\)).

DFS edge classification and applications in connectivity

Above we used gray-to-white edges (tree edges) to construct the DFS tree and gray-to-gray edges (back edges) to detect cycles. But are there other cases such as gray-to-black edges? Well, yes and no – it all depends on the directedness of the graph.

On undirected graphs, DFS classifies each edge into two classes:

• tree edge (father-to-son, gray-to-white): discovers a new white node; part of the recursion tree.
• back edge (descent-to-ancestor, gray-to-gray): rediscovers a gray node; detects a cycle.

Why you can never rediscover a black node? Let’s prove by contradiction. If current node \(v\) rediscovers a black node \(u\) using the edge \((v,u)\), then when \(u\) was still active (gray), it would also visit the \((u,v)\) edge (each edges is bidirectional), so this edge would be either tree edge or back edge, depending on the color of \(v\) at that time.

However, on directed graphs, DFS classifies each edge into four classes:

• tree edge (father-to-son, gray-to-white): discovers a new white node; part of the recursion tree.
• back edge (descendant-to-ancestor, gray-to-gray): rediscovers a gray node. detects a cycle!
• forward edge (ancestor-to-descendant, gray-to-black): rediscovers a black node who is my descendant.
• cross edge (cousin-to-cousin, gray-to-black): rediscovers a black node with no direct ancestry relationship.

As you can see, with these classifications, DFS can detect both undirected and directed cycles. These classifications are also useful in other connectivity/component problems such as detecting strongly connected components in directed graphs and detecting “bridges” in undirected graphs.

You can also use either BFS or DFS to figure out the topological ordering, which we’ll discuss next.