Method and apparatus for decoding data in communication or broadcasting system

A reception device for processing a multiple-in multiple-out (MIMO) signal associated with at least two layers in a communication system, the reception device comprising:

• a receiver;
• and
• a controller coupled with the receiver and configured to:
• decode the MIMO signal, based on at least part of a parity check matrix to determine first low density parity check (LDPC) information bits corresponding to a first layer signal of the MIMO signal,
• identify at least part of first parity bits and second parity bits, and
• determine a second layer signal of the MIMO signal to determine second LDPC information bits corresponding to the second layer signal,
• wherein the second layer signal is determined by removing a signal corresponding to a part of the first LDPC information bits and the at least part of the first parity bits and the second parity bits from the MIMO signal.

A reception device for processing a layered-division multiplexing (LDM) signal generated from two or more layer signals, the reception device comprising:

• a receiver;
• and
• a controller coupled with the receiver and configured to:
• decode the LDM signal, based on at least part of a parity check matrix to determine first low density parity check (LDPC) information bits, first parity bits, and second parity bits corresponding to a first layer signal,
• determine an LDPC syndrome corresponding to the determined first LDPC information bits, first parity bits, and second parity bits,
• determine modified second parity bits, based on the determined second parity bits and the determined LDPC syndrome,
• determine a second layer signal by removing a signal corresponding to the determined first LDPC information bits, the first parity bits, and the modified second parity bits from the LDM signal, and
• decode the second layer signal to determine second LDPC information bits corresponding to the second layer signal,
• wherein the second parity bits correspond to at least part of degree-1 columns in the parity check matrix.