Multimodal: asking about a photograph
How one system can work with a picture and a written question, including accessibility opportunities and risks.
Multimodal case study
What this example shows
See how an image and a written prompt can be combined, and where a generated description may be incomplete.
What goes in
A photograph plus a written or spoken question about it. The image may contain people, locations or private information.
What the AI system does
A multimodal model represents patterns from the image and language together, then generates a response related to the question.
What comes out
A description, answer or suggested action based on what the system identifies in the photograph.
What can go wrong
The system may overlook important details, infer something that is not visible or describe a person inaccurately. Uploading an image can create privacy risks.
Who is responsible
Providers should explain data use and limitations. People should avoid sensitive images and treat descriptions as assistance rather than a complete account.