B picture mode determining method and apparatus in video coding system

A B picture mode determining apparatus of a video coding system comprising:

• a processor performing a program for setting SAD initial values of each mode in consideration of an overhead bit of a 4×4 and 16×16 intra prediction mode, a forward prediction mode, a backward prediction mode, a bi-directional prediction mode and a direct prediction mode, calculating the SAD value of each mode on the basis of the set SAD initial value, and selecting a mode having the smallest SAD value of the calculated SAD values as a mode for a B picture coding,
• wherein the processor sets the SAD initial values of each mode by one of subtracting a certain value from and adding a certain value to the SAD initial set value of each mode in consideration of the overhead bit of each mode the certain value is −16×QP0(QP) for the direct prediction mode and is 24×QP0(QP) for the 4×4 intra prediction mode and, if a difference between a motion vector value calculated from the direct mode and a motion vector value calculated from a true motion is nonacceptable, the QP0(QP) is set so that B pictures are selected as a motion vector value predicted from a true motion, rather than from the direct mode.