# US 2018/0321384 A1 · Systems and methods for generating a structured light depth map with a non-uniform codeword pattern > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2018/0321384 A1 - Document ID: US20180321384A1 - Status: Published application - Kind: A1 - Application number: 15/588,468 - Filed: 2017-05-05 - Published: 2018-11-08 - Earliest claimed: 2017-05-05 - Assignees: QUALCOMM Incorporated - Applicants: QUALCOMM Incorporated - Inventors: Albrecht Johannes Lindner; Kalin Mitkov Atanassov; James Wilson Nash - Web page: https://patentgrep.com/patent/US20180321384A1 ## CPC - G01S 7/4911: Transmitters - G01S 7/4808: {Evaluating distance, position or velocity data} - G01S 17/42: Simultaneous measurement of distance and other co-ordinates (indirect measurement G01S17/46) - G06T 7/521: from laser ranging, e.g. using interferometry; from the projection of structured light - G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract Systems, methods, and devices for generating a depth map of a scene are provided. The method comprises projecting, onto the scene, a codeword pattern including a plurality of light points including a first set of light points projected at a first intensity and a second set of light points projected at a second intensity greater than the first intensity. The method further comprises detecting, from the scene, a reflection of the codeword pattern. The method further comprises generating a first depth map layer at a first resolution. The method further comprises generating a second depth map layer at a second resolution lower than the first resolution. The method further comprises generating the depth map of the scene, wherein the depth map includes depth information for the scene according to each of the first depth map layer and the second depth map layer. ## Claims ### Claim 1 A method for generating a depth map of a scene, comprising: projecting, onto the scene, a codeword pattern including a plurality of light points including a first set of light points projected at a first intensity and a second set of light points projected at a second intensity greater than the first intensity; detecting, from the scene, a reflection of the codeword pattern; generating a first depth map layer at a first resolution; generating a second depth map layer at a second resolution lower than the first resolution; and generating the depth map of the scene, wherein the depth map includes depth information for the scene according to each of the first depth map layer and the second depth map layer. ### Claim 2 (depends on 1) The method of claim 1, wherein projecting the codeword pattern onto the scene comprises transmitting the codeword pattern from a projector, and wherein: the first depth map layer comprises a representation of the scene according to a first distance based on projecting the first set of light points at the first intensity; and the second depth map layer comprises a representation of the scene according to a second distance based on projecting the second set of light points at the second intensity. ### Claim 3 (depends on 2) The method of claim 2, wherein the second set of light points is arranged in a plurality of groups that each include a peak intensity location, the method further comprising: determining one or more light point distances between locations of one or more light points of the first and second sets of light points in the projected codeword pattern and locations of corresponding light points in the reflection of the codeword pattern, wherein generating the first depth map layer is further based on the determined light point distances; and determining one or more group peak distances between peak intensity locations of one or more of the plurality of groups in the projected codeword pattern and corresponding peak intensity locations in the reflection of the codeword pattern, wherein generating the second depth map layer is further based on the determined group peak distances. ### Claim 4 (depends on 3) The method of claim 3, wherein the plurality of light points includes a third set of light points projected at a third intensity greater than the second intensity, the method further comprising: determining one or more additional distances based on locations of the third set of light points in the projected codeword pattern and in the reflection of the codeword pattern; and based on the determined additional distances, generating a third depth map layer at a third resolution, and wherein the depth map includes depth information for the scene according to each of the first, second, and third depth map layers. ### Claim 5 (depends on 3) The method of claim 3, wherein: the projector is located at a first position, and wherein projecting the codeword pattern onto the scene comprises transmitting the codeword pattern from the projector; the receiver is located at a second position, and wherein detecting the reflection of the codeword pattern comprises receiving the codeword pattern at the receiver; the one or more light point distances and the one or more group peak distances are resultants of a difference between the first position and the second position; and generating the first and second depth map layers is based on triangulating codeword positions according to the one or more light point distances and the one or more group peak distances. ### Claim 6 (depends on 1) The method of claim 1, wherein detecting the reflection of the codeword pattern from the scene comprises sensing the reflected codeword pattern at a receiver, wherein the sensing includes: detecting the first set of light points at a first distance from the receiver based on projecting the first set of light points at the first intensity; and detecting the second set of light points at a second distance from the receiver based on projecting the second set of light points at the second intensity. ### Claim 7 (depends on 1) The method of claim 1, wherein projecting the codeword pattern onto the scene comprises transmitting the codeword pattern from a projector electrically coupled to a structured light module, and wherein the projected codeword pattern corresponds to a fixed mask pattern stored at the projector. ### Claim 8 (depends on 1) The method of claim 1, wherein projecting the codeword pattern onto the scene comprises transmitting the codeword pattern from a projector, and wherein: the codeword pattern comprises a total number of light points; a number of light points included in the first set of light points is equal to a first value; a number of light points included in the second set of light points is equal to a second value; a sum of the first value and the second value is equal to the total number of light points; and each of the light points included in the first and second sets of light points are projected from the projector simultaneously. ### Claim 9 An apparatus for generating a depth map of a scene, the apparatus comprising: a projector configured to project, onto the scene, a codeword pattern including a plurality of light points including a first set of light points projected at a first intensity and a second set of light points projected at a second intensity greater than the first intensity; a receiver configured to detect, from the scene, a reflection of the codeword pattern; and a processor configured to: generate a first depth map layer at a first resolution; generate a second depth map layer at a second resolution lower than the first resolution; and generate the depth map of the scene, wherein the depth map includes depth information for the scene according to each of the first depth map layer and the second depth map layer. ### Claim 10 (depends on 9) The apparatus of claim 9, wherein projecting the codeword pattern onto the scene comprises transmitting the codeword pattern from the projector, and wherein: the first depth map layer comprises a representation of the scene according to a first distance based on projecting the first set of light points at the first intensity; and the second depth map layer comprises a representation of the scene according to a second distance based on projecting the second set of light points at the second intensity. ### Claim 11 (depends on 10) The apparatus of claim 10, wherein the second set of light points is arranged in a plurality of groups that each include a peak intensity location, and the processor is further configured to: determine one or more light point distances between locations of one or more light points of the first and second sets of light points in the projected codeword pattern and locations of corresponding light points in the reflection of the codeword pattern, wherein generating the first depth map layer is further based on the determined light point distances; and determine one or more group peak distances between peak intensity locations of one or more of the plurality of groups in the projected codeword pattern and corresponding peak intensity locations in the reflection of the codeword pattern, wherein generating the second depth map layer is further based on the determined group peak distances. ### Claim 12 (depends on 11) The apparatus of claim 11, wherein the plurality of light points includes a third set of light points projected at a third intensity greater than the second intensity, and wherein the processor is further configured to: determine one or more additional distances based on locations of the third set of light points in the projected codeword pattern and in the reflection of the codeword pattern; and based on the determined additional distances, generate a third depth map layer at a third resolution, and wherein generating the depth map of the scene, wherein the depth map includes depth information for the scene according to each of the first, second, and third depth map layers. ### Claim 13 (depends on 11) The apparatus of claim 11, wherein: the projector is located at a first position, and wherein projecting the codeword pattern onto the scene comprises transmitting the codeword pattern from the projector; the receiver is located at a second position, and wherein detecting the reflection of the codeword pattern comprises receiving the codeword pattern at the receiver; the one or more light point distances and the one or more group peak distances are resultants of a difference between the first position and the second position; and generating the first and second depth map layers is based on triangulating codeword positions according to the one or more light point distances and the one or more group peak distances. ### Claim 14 (depends on 9) The apparatus of claim 9, wherein detecting the reflection of the codeword pattern from the scene comprises sensing the reflected codeword pattern at the receiver, wherein the sensing includes: detecting the first set of light points at a first distance from the receiver based on projecting the first set of light points at the first intensity; and detecting the second set of light points at a second distance from the receiver based on projecting the second set of light points at the second intensity. ### Claim 15 (depends on 9) The apparatus of claim 9, wherein the projector is electrically coupled to a structured light module, and wherein the projected codeword pattern corresponds to a fixed mask pattern stored at the projector. ### Claim 16 (depends on 9) The apparatus of claim 9, wherein projecting the codeword pattern onto the scene comprises transmitting the codeword pattern from the projector, and wherein: the codeword pattern comprises a total number of light points; a number of light points included in the first set of light points is equal to a first value; a number of light points included in the second set of light points is equal to a second value; a sum of the first value and the second value is equal to the total number of light points; and each of the light points included in the first and second sets of light points are projected from the projector simultaneously. ### Claim 17 An apparatus for generating a depth map of a scene, the apparatus comprising: means for projecting, onto the scene, a codeword pattern including a plurality of light points including a first set of light points projected at a first intensity and a second set of light points projected at a second intensity greater than the first intensity; means for detecting, from the scene, a reflection of the codeword pattern; means for generating a first depth map layer at a first resolution, wherein the means for generating further generates a second depth map layer at a second resolution lower than the first resolution; and wherein the means for generating further generates the depth map of the scene, wherein the depth map includes depth information for the scene according to each of the first depth map layer and the second depth map layer. ### Claim 18 (depends on 17) The apparatus of claim 17, wherein projecting the codeword pattern onto the scene comprises transmitting the codeword pattern from the means for projecting, and wherein: the first depth map layer comprises a representation of the scene according to a first distance based on projecting the first set of light points at the first intensity; and the second depth map layer comprises a representation of the scene according to a second distance based on projecting the second set of light points at the second intensity. ### Claim 19 (depends on 18) The apparatus of claim 18, wherein the second set of light points is arranged in a plurality of groups that each include a peak intensity location, and the means for generating is further configured to: determine one or more light point distances between locations of one or more light points of the first and second sets of light points in the projected codeword pattern and locations of corresponding light points in the reflection of the codeword pattern, wherein generating the first depth map layer is further based on the determined light point distances; and determine one or more group peak distances between peak intensity locations of one or more of the plurality of groups in the projected codeword pattern and corresponding peak intensity locations in the reflection of the codeword pattern, wherein generating the second depth map layer is further based on the determined group peak distances. ### Claim 20 (depends on 19) The apparatus of claim 19, wherein the plurality of light points includes a third set of light points projected at a third intensity greater than the second intensity, and wherein the means for generating is further configured to: determine one or more additional distances based on locations of the third set of light points in the projected codeword pattern and in the reflection of the codeword pattern; and based on the determined additional distances, generate a third depth map layer at a third resolution, and wherein generating the depth map of the scene, wherein the depth map includes depth information for the scene according to each of the first, second, and third depth map layers. ### Claim 21 (depends on 19) The apparatus of claim 19, wherein: the means for projecting is located at a first position, and projecting the codeword pattern onto the scene comprises transmitting the codeword pattern from the means for projecting; the means for detecting is located at a second position, and detecting the reflection of the codeword pattern comprises receiving the codeword pattern at the means for detecting; the one or more light point distances and the one or more group peak distances are resultants of a difference between the first position and the second position; and generating the first and second depth map layers is based on triangulating codeword positions according to the one or more light point distances and the one or more group peak distances. ### Claim 22 (depends on 17) The apparatus of claim 17, wherein detecting the reflection of the codeword pattern from the scene comprises sensing the reflected codeword pattern at the means for detecting, wherein the sensing includes: detecting the first set of light points at a first distance from the means for detecting based on projecting the first set of light points at the first intensity; and detecting the second set of light points at a second distance from the means for detecting based on projecting the second set of light points at the second intensity. ### Claim 23 (depends on 17) The apparatus of claim 17, wherein the means for projecting comprises a projector electrically coupled to a structured light module, and wherein projecting the codeword pattern onto the scene comprises transmitting the codeword pattern from the means for projecting, and wherein the projected codeword pattern corresponds to a fixed mask pattern stored at the projector. ### Claim 24 (depends on 17) The apparatus of claim 17, wherein projecting the codeword pattern onto the scene comprises transmitting the codeword pattern from the means for projecting, and wherein: the codeword pattern comprises a total number of light points; a number of light points included in the first set of light points is equal to a first value; a number of light points included in the second set of light points is equal to a second value; a sum of the first value and the second value is equal to the total number of light points; and each of the light points included in the first and second sets of light points are projected from the means for projecting simultaneously. ### Claim 25 A non-transitory computer-readable medium comprising code that, when executed, causes a processor of an apparatus to: project, onto a scene, a codeword pattern including a plurality of light points including a first set of light points projected at a first intensity and a second set of light points projected at a second intensity greater than the first intensity; detect, from the scene, a reflection of the codeword pattern; generate a first depth map layer at a first resolution; generate a second depth map layer at a second resolution lower than the first resolution; and generate a depth map of the scene, wherein the depth map includes depth information for the scene according to each of the first depth map layer and the second depth map layer. ### Claim 26 (depends on 25) The medium of claim 25, wherein projecting the codeword pattern onto the scene comprises transmitting the codeword pattern from a projector, and wherein: the first depth map layer comprises a representation of the scene according to a first distance based on projecting the first set of light points at the first intensity; the second depth map layer comprises a representation of the scene according to a second distance based on projecting the second set of light points at the second intensity; and the depth map comprises a representation of the scene according to both the first and second distances. ### Claim 27 (depends on 26) The medium of claim 26, wherein the second set of light points is arranged in a plurality of groups that each include a peak intensity location, and wherein the code, when executed, further causes the processor of the apparatus to: determine one or more light point distances between locations of one or more light points of the first and second sets of light points in the projected codeword pattern and locations of corresponding light points in the reflection of the codeword pattern, wherein generating the first depth map layer is further based on the determined light point distances; and determine one or more group peak distances between peak intensity locations of one or more of the plurality of groups in the projected codeword pattern and corresponding peak intensity locations in the reflection of the codeword pattern, wherein generating the second depth map layer is further based on the determined group peak distances. ### Claim 28 (depends on 27) The medium of claim 27, wherein the plurality of light points includes a third set of light points projected at a third intensity greater than the second intensity, and wherein the code, when executed, further causes the processor of the apparatus to: determine one or more additional distances based on locations of the third set of light points in the projected codeword pattern and in the reflection of the codeword pattern; and based on the determined additional distances, generate a third depth map layer at a third resolution, and wherein generating the depth map of the scene, wherein the depth map includes depth information for the scene according to each of the first, second, and third depth map layers. ### Claim 29 (depends on 27) The medium of claim 27, wherein: the projector is located at a first position, and wherein projecting the codeword pattern onto the scene comprises transmitting the codeword pattern from the projector; detecting the reflection of the codeword pattern comprises receiving the codeword pattern at a receiver located at a second position; the one or more light point distances and the one or more group peak distances are resultants of a difference between the first position and the second position; and generating the first and second depth map layers is based on triangulating codeword positions according to the one or more light point distances and the one or more group peak distances. ### Claim 30 (depends on 25) The medium of claim 25, wherein detecting the reflection of the codeword pattern from the scene comprises sensing the reflected codeword pattern at a receiver, wherein the sensing includes: detecting the first set of light points at a first distance from the receiver based on projecting the first set of light points at the first intensity; and detecting the second set of light points at a second distance from the receiver based on projecting the second set of light points at the second intensity.