Integration of Video Language Models with AI for Filmmaking

A computing system for integrating a video large language model (LLM) with technical inputs for filmmaking, comprising:

• one or more processors;
• and
• one or more memories having stored thereon computer-executable instructions that, when executed, cause the computing system to:
• receive, via one or more processors, metadata related to professional filmmaking techniques and Lidar data generated by a Lidar sensor, the Lidar data including spatial information of objects within a scene;
• process, via one or more processors, the Lidar data and metadata to produce adapted training data;
• and
• apply, via one or more processors, transfer learning techniques to the video LLM using the adapted training data to generate an adapted video LLM configured to generate video content simulating filmmaking techniques based on user input.

A computer-readable medium having stored thereon instructions that when executed by a processor cause a system to perform:

• receive, via one or more processors, metadata related to professional filmmaking techniques and Lidar data generated by a Lidar sensor, the Lidar data including spatial information of objects within a scene;
• process, via one or more processors, the Lidar data and metadata to produce adapted training data;
• and
• apply, via one or more processors, transfer learning techniques to a video LLM using the adapted training data to generate an adapted video LLM configured to generate video content simulating filmmaking techniques based on user input.