# US 10,311,704 B1 · Passenger-related item loss mitigation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,311,704 B1
- Document ID: US10311704B1
- Status: Granted
- Kind: B1
- Application number: 15/939,984
- Filed: 2018-03-29
- Granted: 2019-06-04
- Earliest claimed: 2018-03-29
- Assignees: Futurewei Technologies, Inc.
- Applicants: Futurewei Technologies, Inc.
- Inventors: Qijie Xu; Fatih Porikli; Junqiang Shen
- Web page: https://patentgrep.com/patent/US10311704B1

## CPC

- G06Q 50/40: Business processes related to the transportation industry (shipping G06Q10/083)
- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G06F 16/955: using information identifiers, e.g. uniform resource locators [URL]
- G06F 18/24143: {Distances to neighbourhood prototypes, e.g. restricted Coulomb energy networks [RCEN]}
- G06T 7/11: Region-based segmentation
- G06V 10/454: {Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]}
- G06V 10/764: using classification, e.g. of video objects
- G06V 20/59: inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
- G06V 40/172: {Classification, e.g. identification}
- G08B 21/24: Reminder alarms, e.g. anti-loss alarms
- H04L 51/046: Interoperability with other network applications or services
- B60R 2300/8006: for monitoring and displaying scenes of vehicle interior, e.g. for monitoring passengers or cargo
- G06T 2207/20021: Dividing image into blocks, subimages or windows
- G06T 2207/30268: Vehicle interior
- G08B 13/19647: {Systems specially adapted for intrusion detection in or around a vehicle}
- G08B 29/186: {Fuzzy logic; neural networks}
- H04L 12/1895: {for short real-time information, e.g. alarms, notifications, alerts, updates}
- H04L 51/58: Message adaptation for wireless communication
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G08B: SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- B60R: VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR

## Abstract

A method and device for mitigating passenger-related item loss in a travel-for-hire context are disclosed. An item within a vehicle passenger region is identified, and associated the identified item with a passenger identified by a passenger identifier. A vehicle cabin inventory database is updated with an entry based on the identified item associated with the passenger identifier. A presence of the identified item is detected within the vehicle cabin passenger region when a passenger exit event occurs, and message data is generated including the entry that relates to the identified item based on the presence of the identified item within the vehicle cabin passenger region.

## Claims

### Claim 1

A method of mitigating passenger-related item loss in a travel-for-hire context, the method comprising:
identifying an item within a vehicle cabin passenger region;
associating the identified item with a passenger identified by a passenger identifier;
updating a vehicle cabin inventory database with a database entry based on the identified item associated with the passenger identifier;
in response to an occurrence of a passenger exit event, detecting a presence of the identified item within the vehicle cabin passenger region; and
generating message data including the entry that relates to the identified item based on the presence of the identified item within the vehicle cabin passenger region.

### Claim 2 (depends on 1)

The method of claim 1, wherein the identifying the item comprises:
capturing an image of the vehicle cabin passenger region;
identifying a region-of-interest of the image, wherein the region-of-interest is based on a detection confidence metric relating to the vehicle cabin passenger region;
upsampling the region-of-interest to produce an upsampled region-of-interest, the upsampled region-of-interest to promote image recognition; and
performing image recognition of the upsampled region-of-interest of the image to identify the item within the vehicle cabin passenger region.

### Claim 3 (depends on 2)

The method of claim 2, wherein the capturing the image of the vehicle cabin passenger region occurs upon a passenger entry event.

### Claim 4 (depends on 1)

The method of claim 1, wherein the identifying the item with the vehicle cabin passenger region includes verifying the identified item as a passenger-related item.

### Claim 5 (depends on 1)

The method of claim 1, further comprising verbally announcing the message data via an announcement device.

### Claim 6 (depends on 1)

The method of claim 1, further comprising announcing the message data.

### Claim 7 (depends on 6)

The method of claim 6, wherein the announcing the message data comprises at least one of:
transmitting a text message based on the passenger identifier for receipt by a handheld mobile device;
transmitting the message data based on the passenger identifier for receipt by a client application of the handheld mobile device; or
transmitting the message data based on the passenger identifier as a universal resource locator (URL) link for receipt by a user device.

### Claim 8 (depends on 1)

The method of claim 1, wherein the passenger exit event is based on at least one of:
camera device data; or
vehicle door sensor data.

### Claim 9

A vehicle control unit for in-vehicle tracking of an item, comprising:
a memory containing instructions; and
a processor coupled to the memory, the processor configured to execute the instructions stored in the memory to cause the processor to:
identify an item within a plurality of vehicle cabin passenger regions;
associate the identified item with a passenger identified by a passenger identifier;
update a vehicle cabin inventory database with an entry including the identified item associated with the passenger identifier;
in response to an occurrence of a passenger exit event, detect a presence of the identified item within the plurality of vehicle cabin passenger regions; and
generate message data with the entry relating to the identified item based on the presence of the identified item within the plurality of vehicle cabin passenger regions.

### Claim 10 (depends on 9)

The vehicle control unit of claim 9, comprising further instructions stored in memory to cause the processor to identify the item by:
capturing an image of a vehicle cabin passenger region of the plurality of vehicle cabin passenger regions;
identifying a region-of-interest of the image, wherein the region-of-interest is based on a detection confidence metric relating to the vehicle cabin passenger region of the plurality of vehicle cabin passenger regions;
upsampling the region-of-interest to produce an upsampled region-of-interest, the upsampled region-of-interest to promote image recognition; and
performing image recognition of the upsampled region-of-interest of the image to identify the item within the vehicle cabin passenger region.

### Claim 11 (depends on 9)

The vehicle control unit of claim 9, wherein the identifying the item within the plurality of vehicle cabin passenger regions includes verifying the identified item as a passenger-related item.

### Claim 12 (depends on 9)

The vehicle control unit of claim 9, comprising further instructions stored in memory to cause the processor to verbally announce the message data via a user interface.

### Claim 13 (depends on 9)

The vehicle control unit of claim 9, comprising further instructions stored in the memory to cause the processor to transmit the message data by at least one of:
a text message based on the passenger identifier for receipt by a handheld mobile device;
a client application based on the passenger identifier for receipt by a client application of the handheld mobile device; or
a universal resource locator (URL) link for receipt by a user device.

### Claim 14 (depends on 9)

The vehicle control unit of claim 9, wherein the passenger exit event is based on at least one of:
camera device data; or
vehicle door sensor data.

### Claim 15 (depends on 9)

The vehicle control unit of claim 9, wherein the passenger identifier comprises at least one of:
a username;
an international mobile equipment identity (IMEI) number; or
an electronic serial number (ESN).

### Claim 16

A method of mitigating passenger-related item loss, the method comprising:
identifying, via image recognition, an item within vehicle cabin passenger regions as a passenger-related item;
associating the item with a passenger identified by a passenger identifier when the item is a passenger-related item;
updating a vehicle cabin inventory database entry with the passenger identifier and the passenger-related item;
detecting, in response to an occurrence of a passenger exit event, a presence of the passenger-related item within the vehicle cabin passenger regions;
generating message data based on the presence of the passenger-related item within the vehicle cabin passenger regions and based on the vehicle cabin inventory database entry; and
transmitting the message data for alerting the passenger identified by the passenger identifier.

### Claim 17 (depends on 16)

The method of claim 16, wherein the identifying the item comprises:
capturing an image of the vehicle cabin passenger regions;
identifying a region-of-interest of the image, wherein the region-of-interest is based on a detection confidence metric relating to within the vehicle cabin passenger regions;
upsampling the region-of-interest to produce an upsampled region-of-interest, wherein the upsampled region-of-interest provides an enhanced focus to promote image recognition; and
performing the image recognition of the upsampled region-of-interest of the image to identify the item within the vehicle cabin passenger regions.

### Claim 18 (depends on 16)

The method of claim 16, wherein the transmitting the message data comprises at least one of:
transmitting a text message based on the passenger identifier for receipt by a handheld mobile device;
transmitting the message data based on the passenger identifier for receipt by a client application of the handheld mobile device; or
transmitting the message data based on the passenger identifier as a universal resource locator (URL) link for receipt by a user device.

### Claim 19 (depends on 16)

The method of claim 16, wherein the passenger exit event is based on at least one of:
camera device data; or
vehicle door sensor data.

### Claim 20 (depends on 16)

The method of claim 16, wherein the passenger identifier comprises at least one of:
a username;
an international mobile equipment identity (IMEI) number; or
an electronic serial number (ESN).