# US 10,120,832 B2 · Direct access to local memory in a PCI-E device > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,120,832 B2 - Document ID: US10120832B2 - Status: Granted - Kind: B2 - Application number: 14/721,009 - Filed: 2015-05-26 - Published: 2015-12-03 - Granted: 2018-11-06 - Earliest claimed: 2014-05-27 - Assignees: Mellanox Technologies, Ltd. - Applicants: Mellanox Technologies Ltd. - Inventors: Shachar Raindel; Idan Burstein; Noam Bloch; Shlomo Raikin - Web page: https://patentgrep.com/patent/US10120832B2 ## CPC - G06F 15/17331: {Distributed shared memory [DSM], e.g. remote direct memory access [RDMA]} - G06F 13/28: using burst mode transfer, e.g. direct memory access {DMA}, cycle steal (G06F13/32 takes precedence) - G06F 13/4221: {being an input/output bus, e.g. ISA bus, EISA bus, PCI bus, SCSI bus} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A method includes communicating between at least first and second devices over a bus in accordance with a bus address space, including providing direct access over the bus to a local address space of the first device by mapping at least some of the addresses of the local address space to the bus address space. In response to indicating, by the first device or the second device, that the second device requires to access a local address in the local address space that is not currently mapped to the bus address space, the local address is mapped to the bus address space, and the local address is accessed directly, by the second device, using the mapping. ## Claims ### Claim 1 A method, comprising: allocating, to a first device coupled to a bus that operates in accordance with a bus address space, multiple bus addresses of the bus address space; providing direct access over the bus to a local address space of the first device by defining a mapping between at least some of the addresses of the local address space and the allocated bus addresses of the bus address space; in response to detecting that a second device requires to access a given local address in the local address space of the first device that is not currently mapped by the defined mapping to any of the allocated bus addresses, modifying the defined mapping to also map the given local address of the first device to a respective bus address among the allocated bus addresses; and accessing the given local address of the first device directly, by the second device, using the modified mapping of the defined mapping. ### Claim 2 (depends on 1) The method according to claim 1, wherein modifying the mapping comprises re-mapping one or more addresses of the local address space that were previously mapped to the allocated bus addresses of the bus address space. ### Claim 3 (depends on 1) The method according to claim 1, wherein accessing the given local address comprises translating an allocated bus address in the bus address space that is mapped to the given local address by the modified mapping. ### Claim 4 (depends on 1) The method according to claim 1, wherein the second device uses a second address space that includes a second address corresponding to a respective intermediate address in the bus address space that is mapped to the given local address, wherein accessing the given local address comprises translating the second address to the intermediate address and translating the intermediate address to the given local address using the modified mapping. ### Claim 5 (depends on 4) The method according to claim 4, wherein translating at least one of the second address and the intermediate address comprises holding a respective mapping table. ### Claim 6 (depends on 4) The method according to claim 4, wherein translating at least one of the second address and the intermediate address comprises requesting translation from an address translation service (ATS). ### Claim 7 (depends on 1) The method according to claim 1, wherein accessing the given local address comprises sending, over the bus, a command from the first device to the second device indicating that the given local address is mapped within the modified mapping and ready for access. ### Claim 8 (depends on 1) The method according to claim 1, wherein the second device comprises a network adapter that communicates with a peer network adapter over a communication network, wherein the peer network adapter communicates with the first device using remote direct memory access (RDMA). ### Claim 9 (depends on 1) The method according to claim 1, wherein the first device comprises a first local memory that is accessed using the local address space, and the second device comprises a second local memory, wherein accessing the given local address comprises transferring data directly between the first local memory and the second local memory. ### Claim 10 Apparatus, comprising: first and second devices, which are configured to communicate over a bus in accordance with a bus address space, wherein multiple bus addresses of the bus address space are allocated to the first device, wherein the first device is configured to provide direct access over the bus to a local address space by defining a mapping between at least some of the addresses in the local address space and the allocated bus addresses of the bus address space, and wherein, in response to detecting by the first device or the second device that the second device requires to access a given local address in the local address space of the first device that is not currently mapped by the defined mapping to any of the allocated bus addresses, the first device is configured to modify the mapping to also map the given local address to a respective bus address among the allocated bus addresses, and the second device is configured to access the given local address directly, using the modified mapping of the defined mapping. ### Claim 11 (depends on 10) The apparatus according to claim 10, wherein the first device is configured to modify the mapping by re-mapping one or more addresses of the local address space that were previously mapped to the allocated bus addresses of the bus address space. ### Claim 12 (depends on 10) The apparatus according to claim 10, wherein the first device is configured to translate an allocated bus address in the bus address space that is mapped to the given local address by the modified mapping. ### Claim 13 (depends on 10) The apparatus according to claim 10, wherein the second device uses a second address space that includes a second address corresponding to a respective intermediate address in the bus address space that is mapped to the given local address, wherein the second device is configured to access the given local address by translating the second address to the intermediate address, and wherein the first device is configured to translate the intermediate address to the given local address using the modified mapping. ### Claim 14 (depends on 13) The apparatus according to claim 13, wherein at least one of the first device and the second device is configured to translate the second address or the intermediate address, respectively, by holding a respective mapping table. ### Claim 15 (depends on 13) The apparatus according to claim 13, wherein at least one of the first device and the second device is configured to translate the second address or the intermediate address, respectively, by requesting translation from an address translation service (STA). ### Claim 16 (depends on 10) The apparatus according to claim 10, wherein the first device is configured to send a command to the second device, over the bus, indicating that the given local address is mapped within the modified mapping and ready for access. ### Claim 17 (depends on 10) The apparatus according to claim 10, wherein the second device comprises a network adapter that communicates with a peer network adapter over a communication network, wherein the peer network adapter communicates with the first device using remote direct memory access (RDMA). ### Claim 18 (depends on 10) The apparatus according to claim 10, wherein the first device comprises a first local memory that is accessed using the local address space, and the second device comprises a second local memory, wherein the first and second devices are configured to transfer data directly between the first local memory and the second local memory. ### Claim 19 A method, comprising: communicating between a first device and a second device over a bus that operates in accordance with a bus address space, and allocating for the first device multiple bus addresses of the bus address space, wherein the first device stores data in a storage area in accordance with a logical address space and provides direct access over the bus to a local memory of the first device by defining a mapping between at least some of the addresses of the local memory and the allocated bus addresses of the bus address space; in response to indicating, by the second device, that the second device requires to access a logical address that stores requested data that resides in the storage area but is not currently available in the local memory, fetching the requested data from the logical address to a local address of the local memory that is mapped within the mapping to a respective bus address allocated in the bus address space; and accessing the local address directly, by the second device, using the mapping. ### Claim 20 (depends on 19) The method according to claim 19, wherein fetching the requested data comprises indicating to the second device a physical address of the bus address space corresponding to the local address, when the requested data is available for read in the local address. ### Claim 21 (depends on 19) The method according to claim 19, wherein fetching the requested data comprises writing, prior to fetching the requested data, data that is currently stored in the local address to the storage area. ### Claim 22 (depends on 19) The method according to claim 19, wherein defining the mapping comprises fetching data from a given logical address of the storage area to the local address and associating between the local address and the given logical address. ### Claim 23 (depends on 19) The method according to claim 19, wherein the first device comprises a storage device, and wherein the storage area comprises a nonvolatile memory. ### Claim 24 Apparatus, comprising: first and second devices, which are configured to communicate over a bus in accordance with a bus address space, wherein multiple bus addresses of the bus address space are allocated for the first device, wherein the first device is configured to store data in a storage area in accordance with a logical address space and to provide direct access over the bus to a local memory, by defining a mapping between at least some of the addresses of the local memory and the allocated bus addresses of the bus address space, and wherein, in response to indicating, by the second device, that the second device requires to access a logical address that stores requested data that resides in the storage area but is not currently available in the local memory, the first device is configured to fetch the requested data from the logical address to a local address of the local memory that is mapped within the mapping to a respective bus address allocated in the bus address space, and the second device is configured to access the local address directly, using the mapping. ### Claim 25 (depends on 24) The apparatus according to claim 24, wherein the first device is configured to indicate to the second device a physical address of the bus address space corresponding to the local address, when the requested data is available for read in the local address. ### Claim 26 (depends on 24) The apparatus according to claim 24, wherein the first device is configured to write, prior to fetching the requested data, data that is currently stored in the local address to the storage area. ### Claim 27 (depends on 24) The method according to claim 24, wherein the first device is configured to define the mapping by fetching data from a given logical address of the storage area to the local address and associating between the local address and the given logical address. ### Claim 28 (depends on 24) The method according to claim 24, wherein the first device comprises a storage device, and wherein the storage area comprises a nonvolatile memory.