Method and apparatus for determining a codebook in non-orthogonal multiple access system

A terminal in a wireless communication system, the terminal comprising:

• a transceiver;
• and
• a controller configured to:
• determine activated subcarriers among subcarriers available for use by the terminal based on first bits,
• modulate second bits as a symbol vector based on the activated subcarriers that are determined based on the first bits associated with an index,
• determine a codebook based on information on subcarriers and information on modulation,
• determine a unitary matrix to be assigned to the terminal based on a codeword of the determined codebook, wherein the unitary matrix is determined to maximize a diversity order and shaping gain of the codebook,
• precode the symbol vector based on the codebook and the unitary matrix, and
• transmit the symbol vector to a base station on the subcarriers,
• wherein the second bits are associated with on symbols.

A terminal in a wireless communication system, the terminal comprising:

• a channel encoder configured to encode input bits, wherein the input bits comprise first bits are associated with an index and second bits are associated with symbols;
• an index modulator configured to receive the encoded input bits from the channel encoder, determine activated subcarriers among subcarriers available for use by the terminal based on the first bits, and modulate the second bits as a symbol vector based on the activated subcarriers that are determined based on the first bits associated with the index;
• a processor configured to:
• control to determine a codebook based on information on subcarriers and information on modulation, determine a unitary matrix to be assigned to the terminal based on a codeword of the determined codebook, wherein the unitary matrix is determined to maximize a diversity order and shaping gain of the codebook, and precode the symbol vector based on the codebook and the unitary matrix;
• and
• an orthogonal frequency division multiplexing (OFDM) modulator configured to perform OFDM modulation on the precoded symbol vector to transmit the precoded symbol vector on subcarriers.