Part 3 details the operation of H.264/AVC and discusses issues involved in transmitting video over networks.
By John W. Woods
11.3.8 Video Coder Mode Control
A video coder like H.264/AVC has many coder
modes to be determined. For each macroblock, there is the decision of inter or intra, and quantizer step-size, as well as motion vector blocksize. Each choice (mode) translates into a different number of coded bits for the macroblock. We can write the relevant Lagrangian form as
where Dk and Rk are the corresponding mean-square distortion and rate for block bk when coding in mode mk, and Q is the quantizer step-size parameter. Here Rk must include the bits for the transformed block plus bits for the motion vector(s) and mode decision, the latter usually being negligible for 16 × 16 macroblocks. For the moment assume that the motion vectors have been determined prior to this optimization. Then we sum over all the blocks in the frame to get the total distortion and rate for that frame,
where D and R are the frame's total distortion and rate. Normally the blocks in a frame are scanned in the 2-D raster manner, i.e., left to right and then top to bottom. The joint optimization of all the modes and Q values for these blocks is a daunting problem. In practical coders, often what optimization there is, is done macroblock by macroblock, wherein the best choice of mode mk and Q is done for block bk conditional on the past of the frame in the NSHP sense, resulting in at most a stepwise optimal result. We can achieve this stepwise or greedy optimal result by evaluating Jk in (11.3-2) for all the modes and Q and then choosing the lowest value. This point will then be on the optimized D–R curve for some rate. To generate the entire curve, we sweep through the parameter &lambdamode. Now, in the test model for H.263, an experimentally observed relation is used, that has been theoretically motivated in the high-rate Gaussian case [39],
with the value c = 0.85, approximately on the experimental D–R curve. Therefore, the blockwise optimization in the test model for H.263 then becomes: for each value of quantization parameter Q, choose the mode mk such that
A somewhat different approximation for &lambdamode= ƒ(Q) is used in the H.264/AVC test model. In both cases, this then results in a sequence of macroblocks that is optimized in the so-called constant-slope sense. To actually get a CBR coded sequence, some kind of further rate control has to be applied to decide what value of Q to use for each macroblock in each frame. An easy VBR case results from the choice of constant Q, but then the total bitrate is unconstrained. Also, in practice, different Q values are used for I, P, and B slices or frames, with step-size increasing in a fixed manner. This choice also is usually fixed and not optimized over. Extension of this method to include the needed optimization of the motion vector bitrate in a variable blocksize motion field is contained in Sullivan and Wiegand [39].
11.3.9 Network Adaptation
In H.264/AVC, there is a separation of the video coding layer (VCL), which we have been discussing, from the actual transport bitstream. As in MPEG 4 video, H.264/AVC is intended to be carried on many different networks with different packet or cell sizes and qualities of service. So MPEG simply provides a network abstraction layer (NAL) specification about how to adapt the output of the VCL to any of several transport streams, including transport onMPEG 2, ATM, and IP networks. (In Chapter 12, we discuss further some basic issues in network video.)
Related articles
Part 4 looks at Wavelet codecs.
Printed with permission from Newnes Press, a division of Elsevier. Copyright 2006. "Multidimensional Signal, Image, and Video Processing and Coding" By John Woods. For more information about this title and other similar books, please visit www.elsevierdirect.com.
