# US 10,311,016 B2 · Hidden surface removal in graphics processing systems > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,311,016 B2 - Document ID: US10311016B2 - Status: Granted - Kind: B2 - Application number: 15/607,919 - Filed: 2017-05-30 - Published: 2018-12-06 - Granted: 2019-06-04 - Earliest claimed: 2017-05-30 - Assignees: Arm Limited - Applicants: ARM Limited - Inventors: Frode Heggelund; Toni Viki Brkic; Reimar Gisbert Döffinger - Web page: https://patentgrep.com/patent/US10311016B2 ## CPC - G06F 15/7882: {for self reconfiguration} - G06T 15/405: {using Z-buffer} - G06T 1/20: Processor architectures; Processor configuration, e.g. pipelining - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract A graphics processing pipeline includes a rasteriser, an early culling tester, a renderer, a late culling tester, and a culling test data buffer that stores data values for use by the early and late culling testers. The testing of fragments by the early and late culling testers is controlled in accordance with a first set of state information indicative of when a culling test operation to be used to determine whether to cull the fragments is to be performed, and a second set of state information indicative of when to determine whether to update the culling test data buffer with data for the fragments based on a culling test operation, allocated to the fragments. ## Claims ### Claim 1 A method of operating a graphics processing pipeline that includes: a rasteriser that rasterises input primitives to generate graphics fragments, each graphics fragment having one or more sampling points associated with it; a renderer that processes fragments generated by the rasteriser to generate output fragment data; an early culling tester configured to perform an early culling test on fragments generated by the rasteriser before the fragments are sent to the renderer for processing; a late culling tester configured to perform a late culling test on fragments that have been processed by the renderer before the fragments are output by the renderer to provide output fragment data for sampling points associated with the fragments; and a culling test data buffer configured to store data values for use by the early and late culling testers; the method comprising: providing for a graphics fragment generated by the rasteriser: a first set of state information indicative of whether to perform an early culling test or a late culling test operation to determine whether to cull the fragment; and a second, separate, set of state information indicative of whether to update the culling test data buffer with data for the fragment based on an early culling test or a late culling test operation; and the graphics processing pipeline: subjecting the fragment to an early culling test or a late culling test operation in accordance with the first set of state information provided for the fragment to determine whether to cull the fragment or not; and determining whether to update the culling test data buffer with data for the fragment based on an early culling test or a late culling test operation in accordance with the second set of state information provided for the fragment. ### Claim 2 (depends on 1) The method of claim 1, comprising providing first and second sets of state information for a graphics draw call that is to be used in common for all primitives and fragments of the draw call. ### Claim 3 (depends on 1) The method of claim 1, wherein the first and second sets of state information indicate whether a fragment should be subjected to an early culling test or a late culling test in dependence on at least one condition. ### Claim 4 (depends on 1) The method of claim 1, wherein the first and second sets of state information indicate whether a fragment should be subjected to an early culling test by the early culling tester without waiting for any dependency on the processing of another fragment to be cleared. ### Claim 5 (depends on 1) The method of claim 1, wherein when an early culling test is performed for a fragment that has a dependency on the processing of another fragment without waiting for that dependency to be cleared, the early culling test is performed in a conservative manner and to the extent possible, so as to: in the case of an early culling test to determine whether to cull the fragment, only determine to cull the fragment when it can be determined by the early culling test with the necessary certainty that the fragment will be occluded; and in the case of an early culling test to determine whether to update the culling test data buffer with data for the fragment, only determine to update the culling test data buffer when it can be determined by the early culling test with the necessary certainty that the fragment will not be occluded. ### Claim 6 (depends on 1) The method of claim 1, wherein the first and second sets of state information indicate whether to subject a fragment to an early culling test when it has no dependency on the processing of another fragment, but to a late culling test when it does have a dependency on the processing of another fragment. ### Claim 7 (depends on 1) The method of claim 1, comprising the early culling tester using information received from another early culling test of the graphics processing pipeline when performing an early culling test on fragments generated by the rasteriser. ### Claim 8 (depends on 1) The method of claim 1, further comprising: selecting whether to perform an early culling test or a late culling test operation to determine whether to cull a fragment, and selecting whether to update the culling test data buffer with data for a fragment based on an early culling test or a late culling test operation, based on graphics processing operations that are to be performed when processing the fragment. ### Claim 9 (depends on 8) The method of claim 8, comprising selecting whether to perform an early culling test or a late culling test operation to determine whether to cull a fragment, and selecting whether to update the culling test data buffer with data for a fragment based on an early culling test or a late culling test operation, based on at least one of: operations of a shader program that is to be executed when processing the fragment; whether the processing of the fragment is being used for occlusion query purposes; whether the rendered fragment data for a fragment is to be blended with other rendered fragment data; any application specified culling test operations; and the source or sources of the culling test data values to be used when testing a fragment. ### Claim 10 (depends on 1) The method of claim 1, further comprising: when at least one sampling point associated with a fragment passes an early culling test, sending the fragment onwards for processing and determining if the processing of another sampling point that is in the graphics processing pipeline can be stopped as a consequence of the at least one sampling point passing the early culling test. ### Claim 11 A graphics processing system, comprising: a graphics processing pipeline comprising: a rasteriser that rasterises input primitives to generate graphics fragments, each graphics fragment having one or more sampling points associated with it; a renderer that processes fragments generated by the rasteriser to generate output fragment data; an early culling tester configured to perform an early culling test on fragments generated by the rasteriser before the fragments are sent to the renderer for processing; a late culling tester configured to perform a late culling test on fragments that have been processed by the renderer before the fragments are output by the renderer to provide output fragment data for sampling points associated with the fragments; and a culling test data buffer configured to store data values for use by the early and late culling testers; the graphics processing system further comprising: state information providing processing circuitry configured to provide for a graphics fragment generated by the rasteriser: a first set of state information indicative of whether to perform an early culling test or a late culling test operation to determine whether to cull the fragment; and a second, separate, set of state information indicative of whether to update the culling test data buffer with data for the fragment based on an early culling test or a late culling test operation; and the graphics processing pipeline further being configured to: subject a fragment to an early culling test or a late culling test operation in accordance with a first set of state information provided for the fragment to determine whether to cull the fragment or not; and determine whether to update the culling test data buffer with data for a fragment based on an early culling test or a late culling test operation in accordance with a second set of state information provided for the fragment. ### Claim 12 (depends on 11) The system of claim 11, wherein the state information providing processing circuitry is configured to provide first and second sets of state information for a graphics draw call that is to be used in common for all primitives and fragments of the draw call. ### Claim 13 (depends on 11) The system of claim 11, wherein the first and second sets of state information indicate whether a fragment should be subjected to an early culling test or a late culling test in dependence on at least one condition. ### Claim 14 (depends on 11) The system of claim 11, wherein the first and second sets of state information indicate whether a fragment should be subjected to an early culling test by the early culling tester without waiting for any dependency on the processing of another fragment to be cleared. ### Claim 15 (depends on 11) The system of claim 11, wherein the early culling tester is configured to, when an early culling test is performed for a fragment that has a dependency on the processing of another fragment without waiting for that dependency to be cleared: perform the early culling test in a conservative manner and to the extent possible, so as to: in the case of an early culling test to determine whether to cull the fragment, only determine to cull the fragment when it can be determined by the early culling test with the necessary certainty that the fragment will be occluded; and in the case of an early culling test to determine whether to update the culling test data buffer with data for the fragment, only determine to update the culling test data buffer when it can be determined by the early culling test with the necessary certainty that the fragment will not be occluded. ### Claim 16 (depends on 11) The system of claim 11, wherein the first and second sets of state information indicate whether to subject a fragment to an early culling test when it has no dependency on the processing of another fragment, but to a late culling test when it does have a dependency on the processing of another fragment. ### Claim 17 (depends on 11) The system of claim 11, wherein the early culling tester is configured to use information received from another early culling test of the graphics processing pipeline when performing an early culling test on fragments generated by the rasteriser. ### Claim 18 (depends on 11) The system of claim 11, wherein the state information providing processing circuitry is configured to: select whether to perform an early culling test or a late culling test operation to determine whether to cull a fragment, and select whether to update the culling test data buffer with data for a fragment based on an early culling test or a late culling test operation, based on graphics processing operations that are to be performed when processing the fragment. ### Claim 19 (depends on 18) The system of claim 18, wherein the state information providing processing circuitry is configured to select whether to perform an early culling test or a late culling test operation to determine whether to cull a fragment is to be performed, and to select whether to update the culling test data buffer with data for a fragment based on an early culling test or a late culling test operation, based on at least one of: operations of a shader program that is to be executed when processing the fragment; whether the processing of the fragment is being used for occlusion query purposes; whether the rendered fragment data for a fragment is to be blended with other rendered fragment data; any application specified culling test operations; and the source or sources of the culling test data values to be used when testing a fragment. ### Claim 20 (depends on 18) The system of claim 18, wherein the early culling tester is configured to: when at least one sampling point associated with a fragment passes an early culling test, send the fragment onwards for processing, and provide information for determining if the processing of another sampling point that is in the graphics processing pipeline can be stopped as a consequence of the at least one sampling point passing the early culling test to at least one other stage of the graphics processing pipeline. ### Claim 21 A non-transitory computer readable storage medium storing computer software code which when executing on at least one processor performs a method of setting state information for graphics fragments to be processed by a graphics processing pipeline, the method comprising: for at least one graphics fragment to be processed: determining from requests received for graphics processing operations from an application that requires graphics processing by the graphics processing pipeline: whether to perform an early culling test or a late culling test operation to determine whether to cull the at least one graphics fragment; and whether to update a culling test data buffer using data for the at least one graphics fragment based on an early culling test or a late culling test operation; and providing for the at least one graphics fragment: a first set of state information indicative of whether to perform an early culling test or a late culling test operation selected to determine whether to cull the at least one fragment; and a second, separate, set of state information indicative of whether to update the culling test data buffer with data for the fragment based on an early culling test or a late culling test operation.