# US 2001/0024233 A1 · Camera control system, camera server, camera client, control method, and storage medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2001/0024233 A1
- Document ID: US20010024233A1
- Status: Published application
- Kind: A1
- Application number: 08/950,036
- Filed: 1997-10-14
- Published: 2001-09-27
- Earliest claimed: 1996-10-15
- Assignees: Canon Kabushiki Kaisha
- Inventors: SHINYA URISAKA; TOMOAKI KAWAI; KAZUKO SUZUKI; KOICHI YANO
- Web page: https://patentgrep.com/patent/US20010024233A1

## CPC

- H04N 7/181: {for receiving images from a plurality of remote sources}
- H04N 23/631: {Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters}
- H04N 23/633: {for displaying additional information relating to control or operation of the camera}
- H04N 23/661: Transmitting camera control signals through networks, e.g. control via the Internet
- H04N 23/695: Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

In a system in which a plurality of cameras and microphones, the image pickup directions and audio input directions of which can be remote-controlled, are arranged on a network, when an image pickup inhibited region is set, and when a network client remote-controls a certain camera to pickup a range including the image pickup inhibited region, a portion corresponding to the image pickup inhibited region is not displayed, and sound corresponding to the region is not output.

## Claims

### Claim 1

A camera control system, which has a camera, an angle of which can be controlled, is connected to a predetermined network, and transfers a pickup image by the camera to clients on the network to display the image on display screens of the client devices, comprising:
storage means for storing information indicating levels in units of clients on the network, and information indicating image pickup inhibited regions for substantially inhibiting pickup image in units of levels;
detection means for detecting a level of the connected client by referring to said storage means;
determination means for determining if an image pickup range includes the image pickup inhibited region corresponding to the level of the client; and
inhibition means for, when said determination means determines that the image pickup range includes the image pickup inhibited region, inhibiting image data corresponding to the image pickup inhibited region in the image to be picked up from being transferred.

### Claim 2 (depends on 1)

The system according to claim 1, wherein said inhibition means includes modification means for modifying an image in a region corresponding to the image pickup inhibited region in the pickup image to image quality corresponding to a level of the connected client.

### Claim 3 (depends on 2)

The system according to claim 2, wherein said modification means comprises mask means for masking an image.

### Claim 4 (depends on 2)

The system according to claim 2, wherein said modification means comprises mosaic means for applying a mosaic pattern to an image, and a pixel block size of the mosaic pattern becomes larger for a client with lower level.

### Claim 5

A camera control system, which has a camera, an angle of which can be controlled, is connected to a predetermined network, and transfers a pickup image by the camera to clients on the network to display the image on display screens of the client devices, comprising:
storage means for storing information indicating an image pickup inhibited region;
audio detection means for detecting sound;
determination means for determining if an image pickup range of the camera includes the image pickup inhibited region stored in said storage means; and
inhibition means for, when said determination means determines that the image pickup range includes the image pickup inhibited region, inhibiting image data corresponding to the image pickup inhibited region in the image to be picked up from being transferred, and also inhibiting the sound detected by said audio detection means from being transferred.

### Claim 6 (depends on 5)

The system according to claim 5, wherein said audio detection means includes a microphone, a directivity, detection direction, and input sensitivity of which can be controlled, at least the detection direction being changed in correspondence with an image pickup direction of the camera.

### Claim 7 (depends on 6)

The system according to claim 6, wherein said microphone is mounted on a single panhead together with the camera.

### Claim 8 (depends on 5)

The system according to claim 5, further comprising storage means for storing level information in units of a plurality of clients, and image pickup inhibited regions in units of levels, and
wherein said inhibition means includes modification means for modifying an image in a region corresponding to the image pickup inhibited region in the pickup image to image quality corresponding to a level of the connected client.

### Claim 9 (depends on 8)

The system according to claim 8, wherein said modification means comprises mask means for masking an image.

### Claim 10 (depends on 8)

The system according to claim 8, wherein said modification means comprises mosaic means for applying a mosaic pattern to an image, and a pixel block size of the mosaic pattern becomes larger for a client with lower level.

### Claim 11

A camera control system, which has a camera, an angle of which can be controlled, is connected to a predetermined network, and transfers an image pickup by the camera to clients on the network to display the image on display screens of the client devices, comprising:
storage means for storing information indicating levels in units of clients on the network, and information indicating image pickup inhibited regions for substantially inhibiting pickup image in units of levels;
detection means for detecting a level of the connected client by referring to said storage means;
determination means for determining if an image pickup range includes the image pickup inhibited region corresponding to the level of the client; and
inhibition means for, when said determination means determines that the image pickup range includes the image pickup inhibited region, inhibiting image data corresponding to the image pickup inhibited region in the image to be picked up from being transferred in units of users of the level.

### Claim 12 (depends on 11)

The system according to claim 11, wherein said inhibition means includes modification means for modifying an image in a region corresponding to the image pickup inhibited region in the pickup image to image quality corresponding to a level of the connected client.

### Claim 13 (depends on 12)

The system according to claim 12, wherein said modification means comprises mask means for masking an image.

### Claim 14 (depends on 12)

The system according to claim 12, wherein said modification means comprises mosaic means for applying a mosaic pattern to an image, and a pixel block size of the mosaic pattern becomes larger for a client with lower level.

### Claim 15 (depends on 12)

The system according to claim 12, further comprising audio detection means for detecting sound, and
wherein when said determination means determines that the image pickup range includes the image pickup inhibited region, said inhibition means also inhibits audio information detected by said audio detection means from being transferred.

### Claim 16 (depends on 12)

The system according to claim 12, wherein said audio detection means includes a microphone, a directivity, detection direction, and input sensitivity of which can be controlled, at least the detection direction being changed in correspondence with an image pickup direction of the camera.

### Claim 17

A camera control system, which has a camera, an angle of which can be controlled, is connected to a predetermined network, and transfers an image pickup by the camera to clients on the network to display the image on display screens of the client devices, comprising:
storage means for storing information indicating levels in units of clients on the network, and information indicating image pickup inhibited regions for substantially inhibiting pickup image in units of levels;
detection means for detecting a lowest level among the connected clients by referring to said storage means;
determination means for determining if an image pickup range includes the image pickup inhibited region corresponding to the level detected by said detection means; and
inhibition means for, when said determination means determines that the image pickup range includes the image pickup inhibited region, inhibiting image data corresponding to the image pickup inhibited region in the image to be picked up from being transferred in units of users of the level.

### Claim 18 (depends on 16)

The system according to claim 16, wherein said inhibition means includes modification means for modifying an image in a region corresponding to the image pickup inhibited region in the picked up image to image quality corresponding to the detected lowest level.

### Claim 19 (depends on 17)

The system according to claim 17, wherein said modification means comprises mosaic means for applying a mosaic pattern to an image, and a pixel block size of the mosaic pattern becomes larger for a client with lower level.

### Claim 20 (depends on 16)

The system according to claim 16, further comprising audio detection means for detecting sound, and
wherein when said determination means determines that the image pickup range includes the image pickup inhibited region, said inhibition means also inhibits audio information detected by said audio detection means from being transferred.

### Claim 21 (depends on 20)

The system according to claim 20, wherein said audio detection means includes a microphone, audio detection characteristics including a directivity, detection direction, and input sensitivity of which can be controlled, at least the detection direction being changed in correspondence with an image pickup direction of the camera.

### Claim 22 (depends on 21)

The system according to claim 21, further comprising status information transmission means for transmitting information indicating the audio detection characteristics of said microphone and angle information of the camera to the connected clients, and
wherein each of the clients displays camera and microphone icons on a predetermined map on the basis of the received angle information of the camera and the audio detection characteristic information of said microphone.

### Claim 23

A camera control system, which has a camera, a image pickup direction of which can be changed by remote control from a client on a network, and is constituted by a camera server for transferring an image pickup by the camera, and a client for displaying the transferred image and controlling the image pickup direction of the camera, comprising:
display means for displaying a graphical scroll bar used for change control of the image pickup direction of the camera on a display screen of the client;
storage means for storing information indicating levels in units of clients on the network, and information indicating image pickup inhibited regions corresponding to the levels; and
display control means for distinguishably displaying a portion indicating an image pickup grant range of the camera and a portion indicating an image pickup inhibited range on the basis of the image pickup inhibited region stored in said storage means.

### Claim 24 (depends on 23)

The system according to claim 23, wherein said storage means also stores information indicating levels of the clients, and said display control means distinguishably displays the portion indicating the image pickup grant range of the camera and the portion indicating the image pickup inhibited range in correspondence with the levels of the clients.

### Claim 25 (depends on 23)

The system according to claim 23, wherein said display control means distinguishably displays movable and immovable ranges of rectangular symbols on pan and tilt angle scroll bars for changing an angle.

### Claim 26 (depends on 23)

The system according to claim 23, wherein a plurality of clients can be connected to the camera server.

### Claim 27 (depends on 26)

The system according to claim 26, wherein the camera server comprises means for giving a right of camera access to one connected client, and
means for, when an image pickup by operating the scroll bars by the client given the right of camera access includes the image pickup inhibited region of a client of another level, modifying a portion corresponding to the image pickup inhibited region of the client of the other level in the pickup image not to be substantially displayed, and transferring the modified image to the client of the other level.

### Claim 28

A camera server for controlling a camera in accordance with instructions of an angle and a zoom ratio from a client on a network to pickup an image, and transmitting the pickup image to the client, comprising:
storage means for storing a plurality of regions which define upper limit values of the zoom ratio within a potential image pickup grant region within which pickup image can be done by changing the angle of the camera, the upper limit zoom ratios of the individual regions being different from each other; and
zoom control means for, when the angle of the camera is controlled in accordance with the instructions from the client to pickup an image, controlling an upper limit value of the zoom ratio in a direction of the camera based on the angle not to exceed the upper limit value stored in said storage means.

### Claim 29 (depends on 28)

The server according to claim 28, wherein said storage means stores regions having different patterns and different sizes in units of levels of clients, and
said zoom control means determines an upper limit of the zoom ratio in accordance with the region information corresponding to the level of the client who remote-controls the camera.

### Claim 30

A control method for a camera server for controlling a camera in accordance with instructions of an angle and a zoom ratio from a client on a network to pickup an image, and transmitting the pickup image to the client, comprising:
the storage step of storing, in predetermined storage means, a plurality of regions which define upper limit values of the zoom ratio within a potential image pickup grant region within which pickup image can be done by changing the angle of the camera, the upper limit zoom ratios of the individual regions being different from each other; and
the zoom control step of controlling an upper limit value of the zoom ratio in a direction of the camera based on the angle not to exceed the upper limit value stored in the storage means when the angle of the camera is controlled in accordance with the instructions from the client to pickup an image.

### Claim 31

A client device which is arranged on a network and remote-controls one of a plurality of camera servers each having a camera and microphone via the network, comprising:
output means for receiving an image and sound from the selected camera server, displaying the image, and outputting the sound;
first display means for receiving status information including position information of all the cameras and all the microphones transferred via the network, and displaying icons representing the cameras and microphones on a predetermined map on the basis of the received status information; and
second display means for displaying the icons of the camera and microphone as sources of the image and sound output by said output means to be distinguished from other icons.

### Claim 32 (depends on 30)

The device according to claim 30, wherein the status information includes information indicating image pickup directions of the cameras, and detection directions and detection ranges of the microphones, and
said first and second display means display the camera and microphone icons in accordance with the image pickup direction information and the detection direction information, respectively, and also display each of the microphone icons using two line segments that define the detection range.

### Claim 33

A control method for a client device which is arranged on a network and remote-controls one of a plurality of camera servers each having a camera and microphone via the network, comprising:
the output step of receiving an image and sound from the selected camera server, displaying the image, and outputting the sound;
the first display step of receiving status information including position information of all the cameras and all the microphones transferred via the network, and displaying icons representing the cameras and microphones on a predetermined map on the basis of the received status information; and
the second display step of displaying the icons of the camera and microphone as sources of the image and sound output in the output step to be distinguished from other icons.

### Claim 34

A camera control system for selecting at least one of cameras connected on a network, and transmitting control command information of the camera and audio information near the camera, comprising:
setting means for setting a display inhibition region of an image input from the camera;
display inhibition means for, when a direction range of the display inhibition region set by said setting means overlaps an image pickup range, substantially inhibiting a display of the display inhibition region in the direction range;
audio input means for inputting sound near the camera; and
audio output inhibition means for, when said display inhibition means substantially inhibits the display of the display inhibition region in the direction range, inhibiting the sound input by said audio input means from being output.

### Claim 35 (depends on 34)

The system according to claim 34, further comprising control means for controlling directivity of the sound in accordance with the image input by the camera.

### Claim 36 (depends on 34)

The system according to claim 34, further comprising:
camera index means for indicating at least one of a position and image pickup direction of the camera;
microphone index means for indicating at least one of a position and directivity of said audio input means; and
map display means for displaying a map on which said camera index means and said microphone index means are superimposed.

### Claim 37 (depends on 36)

The system according to claim 36, wherein said setting means sets the display inhibition region of the image on the map display by said map display means.

### Claim 38 (depends on 37)

The system according to claim 37, further comprising user storage means for storing at least one user of said camera control system, and
wherein said setting means can select display inhibition regions of an image in units of users stored in said user storage means.

### Claim 39 (depends on 34)

The system according to any one of claims 34, further comprising:
region storage means for storing the display inhibition region of an image set by said setting means; and
changing means for changing the display inhibition region stored in said region storage means.

### Claim 40 (depends on 34)

The system according to claim 34, wherein said display inhibition means makes a display in the display inhibition region of an image set by said setting means different from a display of an image input by the camera.

### Claim 41 (depends on 35)

The system according to claim 35, wherein the directivity of said audio input means includes at least one of an audio input range, audio input direction, and audio input sensitivity.

### Claim 42 (depends on 41)

The system according to claim 41, wherein said microphone index means that indicates said audio input means, which is inputting sound in communication, is displayed as an index different from other microphone index means.

### Claim 43

A camera server which has an image sensing device and transfers sensed image data to one or a plurality of remote clients, comprising:
storage means for storing groups to which the clients belong, and control limitation information in units of groups;
determination means for determining the group to which the client who has a right of operation to the image sensing device among the connected clients belongs; and
control means for performing camera control in accordance with the control limitation information based on the determination result of said determination means upon reception of a camera control request from the client who has the right of operation.

### Claim 44 (depends on 43)

The server according to claim 43, wherein the control limitation information includes a pan range, tilt range, and zoom range of the image sensing device.

### Claim 45 (depends on 44)

The server according to claim 44, wherein the zoom range varies depending on a view point direction of the image sensing device.

### Claim 46 (depends on 43)

The server according to claim 43, wherein the control limitation information includes a region where a normal display can be done and a region where a resolution is dropped when pan and tilt angles are changed.

### Claim 47

A control method for a camera server which has an image sensing device and transfers sensed image data to one or a plurality of remote clients, comprising:
the determination step of determining, with reference to storage means which stores groups to which the clients belong, and control limitation information in units of groups, the group to which the client who has a right of operation to the image sensing device among the connected clients belongs; and
the control step of performing camera control in accordance with the control limitation information based on the determination result in the determination step upon reception of a camera control request from the client who has the right of operation.

### Claim 48 (depends on 47)

The method according to claim 47, wherein the control limitation information includes a pan range, tilt range, and zoom range of the image sensing device.

### Claim 49 (depends on 48)

The method according to claim 48, wherein the zoom range varies depending on a view point direction of the image sensing device.

### Claim 50 (depends on 47)

The method according to claim 47, wherein the control limitation information includes a region where a normal display can be done and a region where a resolution is dropped when pan and tilt angles are changed.

### Claim 51

A storage medium which stores a program that implements a server for transferring an image sensed by an image sensing device to a remote client in accordance with an instruction from the remote client, storing program codes that implement:
determination means for determining, with reference to storage means which stores groups to which the clients belong, and control limitation information in units of groups, the group to which the client who has a right of operation to the image sensing device among the connected clients belongs; and
control means for performing camera control in accordance with the control limitation information based on the determination result in said determination means upon reception of a camera control request from the client who has the right of operation.

### Claim 52

A camera client connected to a camera server for transferring an image pickup by a camera, comprising:
reception means for receiving data including angle information and image information;
first display means for displaying a current angle state and an angle change range of the camera in accordance with the angle information and angle change allowable range information of image data included in the received angle information; and
second display means for displaying the image data included in the received data.

### Claim 53

A control method for a camera client connected to a camera server for transferring an image pickup by a camera, comprising:
the reception step of receiving data including angle information and image information;
the first display step of displaying a current angle state and an angle change range of the camera in accordance with the angle information and angle change allowable range information of image data included in the received angle information; and
the second display step of displaying the image data included in the received data.

### Claim 54

A storage medium which stores a program that implements a camera client for receiving and displaying an image from a remote camera server, and controlling a camera of the camera server, storing program codes that implement:
determination means for determining, with reference to storage means which stores groups to which the clients belong, and control limitation information in units of groups, the group to which the client who has a right of operation to the image sensing device among the connected clients belongs; and
control means for performing camera control in accordance with the control limitation information based on the determination result in said determination means upon reception of a camera control request from the client who has the right of operation.

### Claim 55

A camera control system for selecting at least one of cameras connected on a network, and transmitting control command information of the camera and audio information near the camera, comprising:
setting means for setting a display inhibition region of an image input from the camera;
display inhibition means for, when a direction range of the display inhibition region set by said setting means overlaps an image pickup range, substantially inhibiting a display of the display inhibition region in the direction range;
audio input means for inputting sound near the camera; and
audio input inhibition means for, when said display inhibition means substantially inhibits the display of the display inhibition region in the direction range, inhibiting said audio input means from inputting the sound.

### Claim 56 (depends on 55)

The system according to claim 55, further comprising control means for controlling directivity of the sound in accordance with the image input by the camera.

### Claim 57 (depends on 55)

The system according to claim 55, further comprising:
camera index means for indicating at least one of a position and image pickup direction of the camera;
microphone index means for indicating at least one of a position and directivity of said audio input means; and
map display means for displaying a map on which said camera index means and said microphone index means are superimposed.

### Claim 58 (depends on 57)

The system according to claim 57, wherein said setting means sets the display inhibition region of the image on the map display by said map display means.
