Method and apparatus of video coding using bi-directional CU weight

A method of video encoding or decoding for a coding system, wherein bi-directional CU (coding unit) weighting (BCW) is enabled for the coding system to allow using multiple pairs of weights, the method comprising:

• receiving input data associated with a current block in a picture;
• generating a Merge candidate list for the current block,
• wherein when a target Merge candidate associated with current bi-directional CU weights for a bi-directional predictor of the current block in the Merge candidate list corresponds to a temporal candidate or a pair-wise candidate, the bi-directional CU weights are not inherited from weights of the temporal candidate and are set to a pre-defined pair of weights;
• when the target Merge candidate associated with current bi-directional CU weights for a bi-directional predictor of the current block in the Merge candidate list corresponds to a spatial candidate, the bi-directional CU weights are inherited from weights of the spatial candidate;
• and
• encoding or decoding the current block or a current motion information of the current block according to the Merge candidate list when Merge mode is selected for the current block.