Back to results
US 12,205,025 B2Granted
Processor memory optimization method and apparatus for deep learning training tasks
AssigneeBeijing Baidu Netcom Science Technology Co., Ltd.
InventorsHaifeng Wang, Xiaoguang Hu, Dianhai Yu
Life of this patent
Estimating the term…
Abstract
The present application discloses a processor video memory optimization method and apparatus for deep learning training tasks, and relates to the technical field of artificial intelligence. In the method, by determining an optimal path for transferring a computing result, the computing result of a first computing unit is transferred to a second computing unit by using the optimal path. Thus, occupying the video memory is avoided, and meanwhile, a problem of low utilization rate of the computing unit of a GPU caused by video memory swaps is avoided, so that training speed of most tasks is hardly reduced.
Claims13
A processor video memory optimization method for deep learning training tasks, comprising:
- receiving, by a first processor, a request instruction input by a user, wherein the request instruction is configured to request training of a deep learning model;
- determining, by the first processor, an optimal path from a first path and a second path, wherein in the first path, a computing result of a first computing unit directly reaches a second computing unit from the first computing unit, and in the second path, the computing result of the first computing unit reaches the second computing unit after a swap operation is executed in a memory of a second processor;
- wherein the first computing unit and the second computing unit are comprised in the first processor, and at least one intermediate computing unit exists between the first computing unit and the second computing unit, and after the computing result of the first computing unit is sequentially processed by the at least one intermediate computing unit, a computing result of a third computing unit is input to the second computing unit, wherein the third computing unit is a computing unit adjacent to the second computing unit and executed before the second computing unit;
- and
- transferring, by the first processor, the computing result of the first computing unit to the second computing unit through the optimal path, wherein the computing result of the first computing unit and the computing result of the third computing unit are used for computation corresponding to the second computing unit;
- wherein when the optimal path is the second path, and after the determining, by the first processor, the optimal path from the first path and the second path, the method further comprises:
- determining, by the first processor, the third computing unit from the at least one intermediate computing unit;
- and
- adding, by the first processor, a dependency relationship between the third computing unit and the swap operation of the second processor.
6 more elements
A processor video memory optimization apparatus for deep learning training tasks, comprising:
- at least one processor;
- and
- a memory communicatively coupled to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to cause the at least one processor to:
- receive a request instruction input by a user, wherein the request instruction is configured to request training of a deep learning model;
- determine an optimal path from a first path and a second path, wherein in the first path, a computing result of a first computing unit directly reaches a second computing unit from the first computing unit, and in the second path, the computing result of the first computing unit reaches the second computing unit after a swap operation is executed in a memory of a second processor;
- wherein the first computing unit and the second computing unit are comprised in the first processor, and at least one intermediate computing unit exists between the first computing unit and the second computing unit, and after the computing result of the first computing unit is sequentially processed by the at least one intermediate computing unit, a computing result of a third computing unit is input to the second computing unit, wherein the third computing unit is a computing unit adjacent to the second computing unit and executed before the second computing unit;
- and
- transfer the computing result of the first computing unit to the second computing unit through the optimal path, wherein the computing result of the first computing unit and the computing result of the third computing unit are used for computation corresponding to the second computing unit;
- wherein when the optimal path is the second path, and after determining the optimal path from the first path and the second path, the at least one processor is caused to:
- determine the third computing unit from the at least one intermediate computing unit;
- and
- add a dependency relationship between the third computing unit and the swap operation of the second processor.
9 more elements
Dependencies are inferred from the claim text.