We propose an optimal framework for active surface extraction from video sequences. An active surface is a collection of active contours in successive frames such that the active contours are constrained by spatial and temporal energy terms. The spatial energy terms impose constraints on the active contour in a given frame. The temporal energy terms relate the active contours in different frames to preserve desired internal and external properties of the active surface. For computational efficiency, we reduce the 3-D active surface ((x, y, t) coordinates) optimization problem to a 2-D model ((ø,t) coordinates) by considering only point indices along normal lines ø of each contour and define the energy terms in a causal way. We develop an n-D dynamic tree programming algorithm to find the optimum of n-D semi-causal functions. We prove that the n-D dynamic tree programming algorithm converges to the global optimum. In particular, the classical 1-D dynamic programming algorithm is a special case of the n-D dynamic tree programming algorithm. The optimal active surface is subsequently obtained by using the 2-D dynamic tree programming algorithm. Simulation results show the efficiency and robustness of the proposed approach in active surface extraction for video tracking and segmentation of the human head in real-world video sequences.