Ripping off the veil: The mysterious PS3 hardware scaler exposed
Introduction
Since its launch last November, Sony's latest gaming platform has given early adopters trouble when attempting to play certain titles in 1080i/p. The console did not automatically upscale its video output to desired resolutions; it was up to either the game software to support these resolutions natively or the internal scalers of users' HDTVs. This forced many people, developers and owners alike, to question the very existence of scaling hardware in the PlayStation 3.
Worse, it created a sour taste in the mouths of owners of older CRT-based HDTV sets, many of which are not capable of accepting a 720p signal at all, and thus only capable of displaying 480i/p and 1080i video signals. If they wanted to run their games in HD resolution, the solution for these disgruntled owners until now was to hope that developers would release their games with 1080i/p support --not a walk in the park for the developer-- or, simply, to buy a new HDTV. As one can imagine, the latter was not the most well-received solution in the history of CE devices.
The key words in that last paragraph would be until now, because with the latest PlayStation 3 software development kit (SDK) update, Sony Computer Entertainment Inc. (SCEI) has finally exposed part of the built-in hardware scaler to developers.
Will this mean that most, if not all, future games will support output at 1080p/i resolutions? Moreover --and this is the question that owners of 1080i-only CRT HDTVs crave to see answered--does this mean that current PS3 games may eventually support the native HD resolution of their televisions? Well, the answer requires some good old-fashioned explanation, so lets start already!
The issue and its solution until now
For openers, let's talk about the way the PlayStation 3 outputs modern videogames. First, the console renders a frame buffer - called the back buffer - in which the drawing passes are executed. Following that, a second frame buffer is rendered, the front buffer. The front buffer is basically the final result of all the rendering operations that took place in the back buffer, and thus it is the picture that one sees on their screen. The front buffer is the optimal point in the display process where scaling solutions can operate.
Before the latest SDK update, developers working on PS3 wishing to support 1080p/i in their 720p-native games had very few options available to them. For 1080i, a theoretical possibility (since its not supported on PS3) would be to do field rendering -- a technique that consists of software scaling the 1280x720 backbuffer into a 1920x540 frontbuffer that will successively render odd and even interlaced lines. The problems with this technique are that it can only output in 1080 interlaced (not progressive), that it has image quality issues, and even more limiting, that it requires the game renderer to refresh at 60Hz (60 frame per seconds) at all times. If the renderer misses a single frame, the output image quality will be terrible. Clearly, field rendering is not an optimal solution for the video output issues of the PlayStation 3.
The only real option available to developers, though, was to upscale the front buffer in software to 1920x1080. This avoids the image quality issues associated with field rendering, and it is also capable of outputting 1080p as well as 1080i signals. Additionally, it is perfectly compatible with games that render at 30Hz, or with unstable framerates (yeah, we know that theres a few of those out there.)
So, you may say, why dont developers just upscale the front buffer to 1080p in software and be done with it? As simple as this sounds, the techniques described above are not free. They have a price in both computational resources and RAM. The computational issue--in the form of doubled fillrate cost--is a problem on its own, since fillrate is not one of RSX's fortes. But its not as stressful as it sounds, since a front buffer is being rendered in a single pass, unlike the back buffer, which requires numerous passes and is therefore more sensitive to resolution increases. The real deal breaker in this scenario is the larger footprint in RAM occupied by these upscaled buffers. It's a price that not all development houses can pay if they plan on their game fitting into the PlayStation 3s memory.
To illustrate how serious the memory problem would be, let's look at the increased requirements that a software upscaler would impose. There are three main resolutions that comprise the common HDTV standards: 720p (1280x720 progressive), 1080i (1920x1080 interlaced) and 1080p (1920x1080 progressive). In order to output at 720p, a PlayStation game needs to render a 1280x720 front buffer; in the case of 1080i/p, it needs a 1920x1080 pixel front buffer. As you can see in the graph below, these different buffers have very different sizes.
As the above graph illustrates, a 1920x1080 front buffer is twice the size of a 1280x720 buffer. With the difference in megabytes measured in the single digits, you could argue that in the grand scheme of things, the difference is not that large. But in reality, when dealing with any closed system, game developers are already trying to shoehorn their games into the available RAM. The PlayStation 3 is no exception, and many developers simply cannot afford to spend more RAM on an upscaled front buffer... and as the games already on the market plainly show, many of them didnt.
In the best case scenario, a developer would wait for RSX to "vblank" (vertical blank--in other words, be left with only the front buffer loaded in memory) and then use RSX to upscale. Developers that seek to have RSX rendering the next frame in the back buffer as soon as possible have to store their front buffer as a 720p image and then upscale it when the GPU is available. In this case, at a single point in time, a 720p back buffer, a 720p front buffer, and a upscaled 1080p frame buffer are all resident in the PlayStation 3 video memory.
Fast forward to the day when a new SDK update was made available, and with it comes a new solution for developers.
The latest SDK: A scaler breaks its chains and is now running free. Well, almost.
And so we arrive at last to the most recent development, the late January PlayStation 3 SDK update. Amongst the newer versions of the various tools included in the SDK lies a new function: the ability for developers to use some of the functionality of the fabled hardware scaler, a scaler many previously doubted existed at all. Interestingly enough, "some" is the key word when describing the unlocked functionality; SCEI only gave access to hardware accelerated horizontal scaling. Horizontal scaling on its own cannot upscale a 720p image into 1080p/i --this would require both horizontal and vertical scaling. Hence, the newly exposed scaler functionality is not enabled in the PS3's user interface directly, but instead will still require developer support to work.
At the time of publication, the reasons why SCEI didnt give developers access to both horizontal and vertical scaling are still unknown, as are the reasons they didnt grant developers access to horizontal scaling until now. The video scaler itself remains shrouded in mystery, as strange as it may seem, but at least now we can say with confidence that it does indeed exist. There are multiple reasons for this continuing secrecy, and insiders are reluctant to discuss them even off the record, nevermind for publication. Nonetheless, the reasons behind SCEI's choices are not our subject today. While a great deal could (and will) be said about the nature of this scaler in the future, today's article will focus on the recently exposed functionality; the details of its hardware and the way that it is integrated into the PlayStation 3s architecture will be reserved for a later article.
Now, lets describe exactly how this added capability works. How can a developer implement a 1080p/i output mode in their 720p-native game for a low cost with this partial scaling? Simply, the developer needs to support one of the new resolutions that SCEI has added to the PlayStation 3s rendering palette. Key among these is new support for a resolution of 960x1080. This results in a framebuffer with relatively few more pixels compared to the more standard 720p, and as a result enjoys a computational cost (fillrate cost) comparable to the one associated with 720p framebuffers to begin with (921,600 pixels for 1280x720 compared to 1,036,800 pixels for 960x1080). At the same time, it benefits from eligibility to be horizontally scaled by the resident hardware scaler.
Additional 1080 rendering modes also supported now include 1280x1080 and 1440x1080. These modes, similar to 960x1080, are capable of being upscaled by the hardware into 1080p/i.
As we can see in this graph, the RAM requirements of a single 960x1080 front buffer are extremely close to a 720p front buffer.
It is worth noting that as of today, support for one of these scalable resolutions is not yet a requirement of new game development; that decision is still in the hands of the developers themselves. However, it is already strongly rumoured that such support will become mandatory through future revisions of the PS3 technical requirements checklist (TRC).
Conclusion
In conclusion, the foretold yet hitherto unseen PlayStation 3 hardware scaler has made its entry onto the development scene with this SDK update. Its appearance should bring hope into the hearts of 1080i-only HDTV owners that all future games may support their television sets natively. They may also look forward to seeing patches to some of the currently released 720p-only games, should developers have the time and inclination to revisit their old code base and hack a new 960x1080 rendering mode into it.
As a postscript, wed like to speculate about another point of debate concerning the PS3 video output capabilities --its lack of VGA resolution support, which itself could also be affected by this latest horizontal revolution. As one would deduce after reading this article, just as with HDTV resolutions, a hypothetical VGA resolution support - notably for 16:10 screens - would entail developer input as well. This would of course require a little more development work for said developers, and additional configurations to run rendering tests on, but the computational and memory costs associated should no longer hinder support of these resolutions in any meaningful way.
Thanks
All the members of the game development community who are always helpful and friendly when I come around bothering them with my pesky questions. Carl and Geo for their tremendous editing help. Stefan, Arun and the B3D Crew. Double thanks to Carl for the late night work I forced on him.
Maison du Cafés coffee, for keeping me awake long enough to fini
http://www.beyond3d.com/articles/ps3scaler/