Can Claude Sonnet 5.5 make videos? What it really does, and how to use it

Scroll through YouTube this month and Claude Sonnet 5.5 video demos are everywhere, with wide-eyed thumbnails and bold claims that the model is now making videos. Animated product teasers, explainer clips, kinetic typography, data that moves. It looks like the model has learned to direct films.

Here is the twist that makes the whole topic easier to understand. Sonnet 5.5 cannot watch a video, and it cannot render one. It never produces a single pixel. What it does, very well, is write the code that a browser plays and a recording tool turns into an MP4. Once you see that, you can use it far more deliberately, and you will know exactly where it will let you down.

If you want to see the results before the explanation, Peter Yang's video Sonnet 5.5 is Here! It's Insane at Making Videos walks through seven examples. Watch it with this article's question in mind: what code is producing each scene?

First, what is Claude Sonnet 5.5?

Anthropic released Claude Sonnet 5.5 on 28 September 2026. It sits in the middle of the Claude 5.5 family: cheaper and faster than Opus 5.5, more capable than Haiku 5.5. Anthropic positions it for everyday work such as fixing bugs and producing polished documents, slides and spreadsheets, and says it generates output more than 30 percent faster than Sonnet 5.

The practical numbers from the model documentation: a 1 million token context window, up to 128,000 output tokens per response, and list pricing of $2 per million input tokens and $10 per million output tokens. It is available through the Claude apps, the Claude API, Amazon Bedrock, Google Cloud and Microsoft Foundry.

The sentence that explains everything: text and images in, text out

The Sonnet 5.5 documentation lists its input and output as "Text and images → text". That one line answers most video questions.

• No video input. You cannot upload an MP4 and ask what happens at the two minute mark. The model has no way to receive it.
• No audio input. Speech, music and sound effects need a separate transcription step before Claude can reason about them.
• No image or video generation. Anthropic's vision guide is explicit that Claude interprets images and cannot create or edit them.
• Animated GIFs do not help. Claude reads only the first frame of an animated image.

So when someone says Sonnet 5.5 made a video, translate it to: Sonnet 5.5 wrote a program, and that program drew the video. That is not a downgrade. Code is precise, editable and repeatable in a way that generated footage is not.

How code becomes a video

Think of a video as a function of time. At 30 frames per second, a 10 second clip is 300 still pictures. If you can write a rule that says what the screen looks like at any moment, a computer can draw all 300 of them. A headline slides in between second 0.5 and second 1.2. A chart bar grows from zero to its value over 800 milliseconds. A logo fades out at second 9.

Language models are good at writing exactly that kind of rule, because it is just code. A typical pipeline looks like this:

1. You describe the scene: length, aspect ratio, frame rate, colours, fonts and what appears on screen when.
2. Sonnet 5.5 writes the animation as HTML, CSS and JavaScript, or in a video framework such as Remotion (React) or Manim (Python).
3. You preview it in a browser or the framework's player and give feedback, ideally with screenshots of the moments that look wrong.
4. A renderer steps through time frame by frame and saves each frame, then a tool such as FFmpeg encodes them into an MP4.
5. Voiceover, music and captions are added in the same composition or in an editor afterwards.

Anthropic's own launch page hints at this style of work. Its demos include generated animated scenes such as starling murmurations, drifting sand dunes and a clock made of clocks, all built as code rather than filmed.

Why Sonnet 5.5 is a good fit for this work

Code-to-video is mostly a loop: write, look, fix, look again. Several published traits of Sonnet 5.5 help that loop.

• Design sense. Early testers quoted by Anthropic credit it with a sharp eye for design and UI polish, which matters when the output is something people watch.
• Speed. Output that arrives over 30 percent faster than Sonnet 5 means more rounds of feedback in the same afternoon.
• Long output. Up to 128,000 output tokens per response is room for a full multi-scene composition in a single answer.
• Large context. A 1 million token window can hold your brand guide, previous scenes and a long feedback history together.
• Visual judgement. Anthropic reports 61.6 percent on the Chartography benchmark without tools, against 15.6 percent for Sonnet 5, and says it is the first Sonnet model to beat Pokémon Red working only from screenshots.

That last point matters more than it sounds. Following a game from screenshots means tracking what changed between one image and the next. That is the same skill you rely on when you show Claude two frames of your animation and ask why the transition feels clumsy.

Teaching it to watch: reviewing video through frames

Claude cannot play a video, but it can look at pictures from one. Pull still frames at a steady interval, send them in order with their timestamps, and ask your question. This works for checking your own rendered animation, reviewing a screen recording of a bug, or summarising a slide-based talk.

# One frame every 2 seconds, scaled to 1280 px wide
ffmpeg -i demo.mp4 -vf "fps=1/2,scale=1280:-2" frames/frame_%03d.jpg

# Optional: pull the audio out so it can be transcribed separately
ffmpeg -i demo.mp4 -vn -ac 1 -ar 16000 audio.wav

Then label each frame with its time so Claude can refer to moments precisely:

import base64, glob
import anthropic

client = anthropic.Anthropic()
content = []
for i, path in enumerate(sorted(glob.glob("frames/*.jpg"))):
    with open(path, "rb") as f:
        data = base64.standard_b64encode(f.read()).decode()
    content.append({"type": "text", "text": f"Frame at {i * 2} s:"})
    content.append({"type": "image", "source": {"type": "base64", "media_type": "image/jpeg", "data": data}})
content.append({"type": "text", "text": "These frames are from a 2 minute product video. List any frame where text is cut off, unreadable or overlapping, and suggest a fix for each."})

response = client.messages.create(model="claude-sonnet-5-5", max_tokens=4000, messages=[{"role": "user", "content": content}])
print(response.content[-1].text)

A few limits from Anthropic's vision guide shape how you sample:

• Up to 600 images per API request for Sonnet 5.5, but only 20 per message in the claude.ai chat app.
• Once a request has more than 20 images, keep each one at or under 2000 pixels on its longest side.
• The whole request must stay under 32 MB, so use JPEG frames or the Files API for long sequences.
• Sampling every second or two catches scenes and on-screen text. It will miss very fast motion between frames.

And the cost is easy to estimate. Claude counts images in 28 by 28 pixel patches. A 1280 by 720 frame is 46 by 26 patches, about 1,196 tokens. Sixty frames, two minutes at one frame every two seconds, comes to roughly 72,000 input tokens. At $2 per million, that is about 14 US cents before your text prompt and the reply.

Your first code-to-video project, step by step

Start with something short and specific: a 15 second teaser for a product or event. Short videos keep feedback rounds quick and mistakes cheap.

1. Write a scene brief before you prompt: duration, aspect ratio, frame rate, colours, fonts, and a second by second list of what appears.
2. Ask for a single self-contained HTML file where everything is driven by one time value, plus a way to jump to any second for checking.
3. Open it in a browser. Screenshot the moments that look wrong and send them back with a specific note, such as "at 4 s the price overlaps the logo".
4. When the preview is right, render it with a capture tool or framework renderer and encode it with FFmpeg.
5. Add voiceover or music you have the rights to use, then watch the final file on a phone, which is where most people will see it.

Create a 15 second, 1080x1920 vertical video as one self-contained HTML file.
Render at 30 fps. Drive every animation from a single time value t in seconds,
and let me jump to any moment with ?t=4.5 in the URL for checking.

Palette: near-black background, white text, one accent colour #7C5CFF.
Font: Inter from Google Fonts.

0.0 to 2.0 s  The words "Your shop, open at 2am" type in, centred.
2.0 to 6.0 s  Three phone screens slide up showing order, payment, delivery.
6.0 to 11.0 s A counter rises from 0 to 120 with the label "orders this week".
11.0 to 15.0 s Logo fades in with "Launching 1 November" beneath it.

Keep all text inside a 90 px safe margin. No external images.

Notice what the brief does. It turns taste into numbers. The clearer your timing and constraints, the less the model has to guess, and guessing is where AI-made videos start to look generic.

Where it falls short

Knowing the limits will save you from disappointing results.

• No real footage. Code is excellent at shapes, type, charts, interfaces and abstract motion. It will not give you a photorealistic person walking down a Nairobi street. That needs a camera or a video generation model.
• No built-in sound. Voiceover needs a text-to-speech service or a recording, and timing it to the animation is still your job.
• It cannot see its own result unless you show it. The model writes code blind. Rendering, watching and sending back frames is what closes the loop.
• Small visual bugs. Fonts that load late, text that overflows on one aspect ratio, easing that feels mechanical. Review the rendered video, not just the code.
• Rights and licences still apply. Check the licence of any framework, font, music or image you use, and do not imply a client endorsed something they did not.

Treat the output like any other generated code. Our guide to reviewing AI generated code applies here too: confirm the result, then read the code that produced it.

What this means if you run a business

The interesting shift is not that a model can make a pretty clip. It is that video becomes something you can version, template and regenerate. Change a price in one place and every product video updates. Feed a spreadsheet in and get a personalised clip per customer. Ship release notes as a 20 second animation instead of a wall of text.

That is a software problem as much as a creative one: templates, data, rendering, storage and review. If you are new to putting a model to work, start small, as we suggest in your first AI agent should have one job. One video type, one template, a person who approves every render.

Our web development team builds websites, online shops and booking systems for businesses. If you want animated product pages or a code-to-video idea explored as part of that work, talk to the Alphonix team. We will tell you honestly whether code-generated video suits your project, or whether a camera and an editor would serve you better.