Two-stage camera-based video search using a combination of draft model and full vision language model (VLM)

An apparatus comprising:

• an interface configured to receive (i) pixel data of an environment and (ii) a user input comprising a natural text description;
• and
• a processor configured to (i) process said pixel data arranged as raw video frames, (ii) perform encoding operations to generate encoded video frames in response to said raw video frames, (iii) implement a first AI module configured to operate on said raw video frames and (iv) upload to a remote computing device (A) image match data in response to said user input and said raw video frames and (B) said encoded video frames, wherein
• (a) said first AI module is configured to (i) parse said natural text description of said user input to determine search parameters, (ii) search said raw video frames for a match between said search parameters and video data of said raw video frames and (iii) enable said upload of said image match data in response to said match,
• (b) said remote computing device is configured to (i) store each of said encoded video frames, (ii) implement a second AI module and (iii) extract a decoded video sequence from only a portion of said encoded video frames identified in said image match data,
• (c) said second AI module implements a full AI model and said first AI module implements a draft model version of said full AI model, and
• (d) said second AI module is configured to generate a plain language description of said decoded video sequence in response to receiving said upload of said image match data.