影片理解¶
影片資料是豐富的資訊來源，可協助您完成任務並理解周遭世界。將 Gemma 與影片資料搭配使用，能幫助您理解空間關係、解讀人類互動，並支援情境感知。本教學示範如何使用 Hugging Face Transformers 開始用 Gemma 處理影片資料。
安裝 Python 套件¶
安裝執行 Gemma 模型與發出請求所需的 Hugging Face 程式庫。
# Install PyTorch & other libraries
pip install torch accelerate
# Install the transformers library
pip install "transformers>=5.10.1"
載入模型¶
使用 transformers 程式庫，透過 AutoProcessor 與 AutoModelForImageTextToText 類別建立 processor 與 model 的實例，如以下程式碼範例所示：
MODEL_ID = "google/gemma-4-E2B-it"
from transformers import AutoProcessor, AutoModelForMultimodalLM
model = AutoModelForMultimodalLM.from_pretrained(MODEL_ID, dtype="auto", device_map="auto")
processor = AutoProcessor.from_pretrained(MODEL_ID)
執行影片資料請求¶
您可以在對話範本中引用檔案位置。processor 接著會負責載入影片並進行處理。以下範例程式碼示範使用對話範本提出請求、生成輸出並解碼回應：
from transformers import AutoProcessor
messages = [
{
'role': 'user',
'content': [
{"type": "video", "video": "https://github.com/bebechien/gemma/raw/refs/heads/main/videos/ForBiggerBlazes.mp4"},
{'type': 'text', 'text': 'Describe this video.'}
]
}
]
inputs = processor.apply_chat_template(
messages,
tokenize=True,
return_dict=True,
return_tensors="pt",
add_generation_prompt=True,
).to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512)
response = processor.decode(outputs, skip_special_tokens=False)
print(response[0])
<bos><|turn>user
00:00 <|image><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><image|> 00:00 <|image><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><image|> 00:00 <|image><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><image|> 00:01 <|image><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><image|> 00:01 <|image><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><image|> 00:02 <|image><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><image|> 00:02 <|image><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><image|> 00:03 <|image><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><image|> 00:03 <|image><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><image|> 00:04 <|image><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><image|> 00:04 <|image><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><image|> 00:05 <|image><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><image|> 00:05 <|image><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><image|> 00:06 <|image><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><image|> 00:06 <|image><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><image|> 00:06 <|image><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><image|> 00:07 <|image><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><image|> 00:07 <|image><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><image|> 00:08 <|image><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><image|> 00:08 <|image><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><image|> 00:09 <|image><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><image|> 00:09 <|image><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><image|> 00:10 <|image><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><image|> 00:10 <|image><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><image|> 00:11 <|image><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><image|> 00:11 <|image><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><image|> 00:12 <|image><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><image|> 00:12 <|image><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><image|> 00:13 <|image><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><image|> 00:13 <|image><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><image|> 00:14 <|image><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><image|> 00:14 <|image><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><|video|><image|>Describe this video.<turn|>
<|turn>model
This video appears to be an advertisement or promotional content for **HBO GO**, highlighting its availability on various devices, including televisions and Chromebooks.
Here's a breakdown of the key moments:
* **00:00 - 00:03:** The initial scenes show a tablet displaying content, likely from *Game of Thrones* (indicated by the visuals and later text). Text overlays mention "Shortened sequences."
* **00:03 - 00:08:** The content shifts to vibrant, intense scenes from *Game of Thrones*, featuring fire, dragons, and characters like Daenerys (implied by the final clips).
* **00:08 - 00:09:** A close-up shot shows a hand interacting with a device, which seems to be a **Chromecast** (indicated by the logo), suggesting casting or streaming functionality.
* **00:09 - 00:10:** The video explicitly states, "**HBO GO right on your TV**," providing details about its availability and access restrictions.
* **00:10 - 00:12:** A concluding message says, "**Everything you love, now on your TV**," followed by a pricing structure: "**For $35. For everyone. Plus tax. HBO's subscription required.**"
* **00:13 - 00:14:** The final screen displays the **Chromecast** logo and the URL `google.com/chromecast`, indicating that the service is accessible via Google Chromecast devices.
**In summary, the video promotes the HBO GO streaming service, emphasizing its presence on TVs and Chromecast devices, and presenting a subscription price.**<turn|>
總結¶
在本指南中，您學會了如何使用 Gemma 4 模型處理影片資料。範例示範了如何使用 processor 自動處理輸入影片，並生成影片內容的文字理解。