Engineering and Developers Blog
What's happening with engineering and developers at YouTube
Smoother <video> in Chrome
Tuesday, November 10, 2015
Video quality matters, and when an HD or HFR playback isn’t smooth, we notice. Chrome noticed. YouTube noticed. So we got together to make YouTube video playback smoother in Chrome, and we call it Project Butter.
For some context, our brains fill in the motion in between frames if each frame is onscreen the same amount of time - this is called
. In other words, a 30 frames per second video won’t appear smooth unless each frame is spaced evenly each 1/30th of a second. Smoothness is more complicated than just this - you can read more about it in this
article by Michael Abrash at Valve
Frame rates, display refresh rates and cadence
Your device’s screen redraws itself at a certain frame rate. Videos present frames at a certain rate. These rates are often not the same. At YouTube we commonly see videos authored at
, 29.97, 30, 48,
, 59.94, and
frames per second (fps) and these videos are viewed on displays with different
- the most common being 50Hz (Europe) and 60Hz (USA).
For a video to be smooth we need to figure out the best, most regular way to display the frames - the best cadence. The ideal cadence is calculated as the ratio of the display rate to frame rate. For example, if we have a 60Hz display (a 1/60 second display interval) and a 30 fps clip, 60 / 30 == 2 which means each video frame should be displayed for two display intervals of total duration 2 * 1/60 second.
We played videos a bunch of different ways and scored them on smoothness.
Using off the shelf
HDMI capture hardware
special video clips
we computed a percentage score based on the number of times each video frame was displayed relative to a calculated optimal display count. The higher the score, the more frames aligned with the optimal display frequency. Below is a figure showing how Chrome 43 performed when playing a 30fps clip back on a 60Hz display:
Smoothness: 68.49%, ~Dropped: 5 / 900 (0.555556%)
The y-axis is the number of times each frame was displayed, while the x-axis is the frame number. As mentioned previously the calculated ideal display count for a 30fps clip on a 60Hz display is 2. So, in an ideal situation, the graph should be a flat horizontal line at 2, yet Chrome dropped many frames and displayed certain frames for as many as 4 display cycles! The smoothness score reflects this - only 68.49 percent of frames were displayed correctly. How could we track down what was going on?
Using some of the
performance tracing tools
built into Chrome, we identified
inherent to the existing design for video rendering as the culprit. These issues resulted in both missed and irregular video frames on a regular basis.
There were two main problems in the interactions between Chrome’s
(responsible for drawing frames) and its media pipeline (responsible for generating frames) --
The compositor didn’t have a timely way of knowing when a video frame needed display. Video frames were selected on the media pipeline thread while the compositor would occasionally come along looking for them on the compositor thread, but if the compositor thread was busy it wouldn’t get the notification on time.
Chrome’s media pipeline didn’t know
the compositor would be ready to draw its next new frame. This led to the media pipeline sometimes picking a frame that was too old by the time the compositor displayed it.
In Chrome 44, we re-architected the media and compositor pipelines to communicate carefully about the intent to generate and display. Additionally, we also improved which video frames to pick by using the optimal display count information. With these changes, Chrome 44 significantly improved on smoothness scores across all video frame rates and display refresh rates:
Smoothness: 99.33%, ~Dropped: 0 / 900 (0.000000%)
Smooth like butter. Read more in
public design document
, if you’re interested in further details.
Dale Curtis, Software Engineer, recently watched
WARNING: SCARIEST GAME IN YEARS | Five Nights at Freddy's - Part 1
Richard Leider, Engineering Manager, recently watched
Late Art Tutorial
Renganathan Ramamoorthy, Product Manager, recently watched
Video Game High School
google developers live
Live Streaming API
media:keywords keywords tags metadata
ssl https certificate staging stage
Super Chat API
YouTube Data API
youtube developers live
YouTube Reporting API
Press & Blog
Creators & Partners
YouTube Creator Blog
YouTube Trends Blog