Method and apparatus of video data processing with restricted block size in video coding

A method of processing video data in a video decoding system, comprising:

• receiving a video bitstream associated with a current coding tree unit (CTU) in a current picture;
• parsing one or more CTU-level syntaxes and residues for the current CTU from the video bitstream;
• determining a size for the current CTU or a current coding unit (CU), wherein the current CTU includes one or more current CUs;
• checking if the size, a width, or a height of the current CTU or the current CU is greater than a threshold;
• if the size, width, or height of the current CTU or the current CU is greater than the threshold, splitting the current CTU or the current CU into multiple blocks until a size, width, or height of each block in the current CTU or the current CU is not greater than the threshold;
• processing the current CTU or the current CU for prediction and inverse transform;
• and
• decoding the current CTU according to the one or more CTU-level syntaxes parsed from the video bitstream.

An apparatus of processing video data in a video decoding system, the apparatus comprising one or more electronic circuits configured to:

• receiving a video bitstream associated with a current coding tree unit (CTU) in a current picture;
• parsing one or more CTU-level syntaxes and residues for the current CTU from the video bitstream;
• determining a size for the current CTU or a current coding unit (CU), wherein the current CTU includes one or more current CUs;
• checking if the size, a width, or a height of the current CTU or the current CU is greater than a threshold;
• if the size, width, or height of the current CTU or the current CU is greater than the threshold, splitting the current CTU or the current CU into multiple blocks until a size, width, or height of each block in the current CTU or the current CU is not greater than the threshold;
• processing the current CTU or the current CU for prediction and inverse transform;
• and
• decoding the current CTU according to the one or more CTU-level syntaxes parsed from the video bitstream.