A method of controlling an image pickup apparatus including a photoelectric conversion element configured to generate and accumulate electric charge according to an optical input, a charge accumulation portion configured to accumulate the electric charge, a charge holding portion disposed between the photoelectric conversion element and the charge accumulation portion, a first transfer switch configured to selectively separate and connect between the photoelectric conversion element and the charge holding portion, and a second transfer switch configured to selectively separate and connect between the charge holding portion and the charge accumulation portion,
- the method comprising setting an operation mode to one of a first operation mode in which the second transfer switch separates the charge holding portion and the charge accumulation portion during a charge accumulation operation in the photoelectric conversion element, and a second operation mode in which the second transfer switch connects between the charge holding portion and the charge accumulation portion during the charge accumulation operation in the photoelectric conversion element, according to a photographing condition for performing photographing of an object.