Display of signing video through an adjustable user interface (UI) element
AssigneeSony Group Corporation
InventorsBrant Candelore, Adam Goldberg, Robert Blanchard
Life of this patent
Estimating the term…
Abstract
An electronic device and method are provided for display of signing video through an adjustable user interface (UI) element. The electronic device receives a first media stream that includes a video. The electronic device determines a location of a signer in the video. The electronic device extracts a video portion that corresponds to the determined location in the video. The electronic device controls a playback of the video on the display device. The electronic device controls the display device based on the playback to render the UI element on the display device and display the extracted video portion inside the UI element.
Claims15
An electronic device, comprising:
- circuitry communicatively coupled to a display device, wherein the circuitry is configured to:
- receive a first media stream that includes a video, wherein the video includes a signer that is one of an animated character or a person who uses a sign language to perform in the video;
- detect, by application of a neural network model on frames of the video, hand signs associated with the sign language in the video;
- determine a location of the signer in the video based on the detection of the hand signs, wherein
- the determined location of the signer comprises image coordinates which correspond to corners of a rectangular region of the video that includes the signer;
- receive a first user input regarding the determined location of the signer in the video;
- extract a video portion from the rectangular region of the video based on the received first user input, wherein the video portion corresponds to the determined location of the signer in the video;
- control a playback of the video on the display device;
- and
- control the display device based on the playback to:
- render a user interface (UI) element on the display device;
- and
- display the extracted video portion inside the UI element.
A method, comprising:
- receiving a first media stream that includes a video, wherein the video includes a signer that is one of an animated character or a person who uses a sign language to perform in the video;
- detecting, by application of a neural network model on frames of the video, hand signs associated with the sign language in the video;
- determining a location of the signer in the video based on the detection of the hand signs, wherein
- the determined location of the signer comprises image coordinates which correspond to corners of a rectangular region of the video that includes the signer;
- receiving a user input regarding the determined location of the signer in the video;
- extracting a video portion from the rectangular region of the video based on the received user input, wherein the video portion corresponds to the determined location of the signer in the video;
- controlling a playback of the video on a display device;
- and
- controlling the display device based on the playback for:
- rendering a user interface (UI) element on the display device;
- and
- displaying the extracted video portion inside the UI element.
A non-transitory computer-readable medium having stored thereon, computer-executable instructions which, when executed by an electronic device, cause the electronic device to execute operations, the operations comprising:
- receiving a first media stream that includes a video, wherein the video includes a signer that is one of an animated character or a person who uses a sign language to perform in the video;
- detecting, by application of a neural network model on frames of the video, hand signs associated with the sign language in the video;
- determining a location of the signer in the video based on the detection of the hand signs, wherein
- the determined location of the signer comprises image coordinates which correspond to corners of a rectangular region of the video that includes the signer;
- receiving a user input regarding the determined location of the signer in the video;
- extracting a video portion from the rectangular region of the video based on the received user input, wherein the video portion corresponds to the determined location of the signer in the video;
- controlling a playback of the video on a display device;
- and
- controlling the display device based on the playback for:
- rendering a user interface (UI) element on the display device;
- and
- displaying the extracted video portion inside the UI element.
Dependencies are inferred from the claim text.