Fast Inter Mode Decision Based on RD Costs and Frequencies of Modes

The new video coding standard H.264/AVC achieves higher coding efficiency than previous standards. However, the efficiency arises at the cost of significant complexity. In this paper, an efficient intermode decision algorithm is proposed to cut down the complexity of an exhaustively full mode decision algorithm in the reference software. Statistical variables including RD (Rate Distortion) costs and frequencies of various modes, correlation of modes between current MB (Macroblock), and their spatial and temporal neighbor MBs are utilized to achieve fast mode decision in the algorithm. The proposed algorithm is composed of three primary steps: The characteristics information of video content, RD cost, and frequency of each mode, are obtained by statistics to assist the decision of the best coding mode for the MB. Because the accuracy of the statistical results will affect the succedent mode decision process, FMD (Full Mode Decision) algorithm are used to get the statistical values from several training frames. In addition, considering the above statistical variables change with video content's changes, they are updated at regular intervals. The best coding modes of current MBs and their neighbor MBs (both spatial and temporal) are highly correlated, so the coding mode of these neighbor MBs are used as an indication to that of current MBs. The difference of RD costs between MB mode (16×16, 16×8 and 8×16) and non-MB modes is used to decide the possible modes class for current MB. Furthermore, the possible modes are prioritized based on their occurring probabilities such that the highest probable mode will be tried first. During this process, the computed RD cost is checked against a content adaptive RD cost threshold to decide if the mode decision process should be terminated before trying the remaining modes in the class. In this way many unlikely coding modes are skipped, and the computational time is significantly reduced. Experimental results show that the proposed algorithm performs well on both the low motion sequences and the high motion sequences due to utilizing the sequence-dependent statistical variables as adaptive threshold to decide the best coding modes. On average, the algorithm reduces the total encoding time by 66.3% while image quality (PSNR) degradation is only 0.154dB with bitrate increase about 0.21%.