Method of manufacturing a semiconductor device comprising a semiconductor body which is provided at a surface with a non-volatile memory comprising a memory cell with a gate structure with an access gate and a gate structure with a control gate and a charge storage region situated between the control gate and the semiconductor body, in which method on the surface of the semiconductor body a first one of said gate structures is formed with side walls extending substantially perpendicular to the surface, a conductive layer is deposited on and next to said first gate-structure, the conductive layer is subjected to a planarizing treatment until the first gate structure is exposed and the so planarized conductive layer is patterned so as to form at least a part of the other gate structure adjoining the first gate structure, characterized in that, to perform said patterning of the planarized conductive layer, the planarized conductive layer is etched back so as to expose an upper portion of the side walls of the first gate structure, a spacer is formed on the exposed upper portion of the side walls of first gate structure and the conductive layer is etched anisotropically using the spacer as a mask.