This is a great write up. I also want to thank you for being a good sport. I chose to use Trading Stuff for my analysis because it is the kind of game I enjoy. I posted an update at the top of my blog post with a link to this.
Ben Garney
· 2 months ago
No problem. I totally understand the lure of reverse engineering things, and I'm a little flattered you chose my app. :) Thanks for the link on your blog!
Emanuele Cipolloni
· 2 months ago
Thanks Ben for highlighting how you have achieved this port, I also got the game because I liked the concept, I'm not criticizing the game concept itself.
However, I don't get why a bad performance of the actual Flash software renderer is to blame on the "crappy ARM11" chip or the "very slow ram", iPhone is not designed and sold with the "Made For Flash" sticker, Apple give plenty of resources (may I add free?) to develop properly on the iPhone and it is not really necessary to get down to ARM assembler in order get smooth 3D games at high frame rates or any other application.
It would be worth noting also that the converted Flash app is always HW accelerated as it uses the 3D chip to stamp texture on the screen but it actually uses it in the wrong way. When you use the "cache AsSurface" flag on a sprite you're not activating any special hardware setting in the iPhone side, you're only telling to the Flash soft-renderer to consider an already rendered object as static, in this way, the texture associated to a certain sprite is not rendered again, but for this to work, most if not all display list objects have to be static or very small to avoid the bottleneck in texture transfer from ram to 3D chip.
Nobody doubt that Adobe teams are full of very talented people, just not sure how focused they are, I bought my CS4 over 1 year ago and I feel kind of frustrated that they jump and down to show CS5 when this stuff that costed me over 2000 euros still crash like it is a beta and nobody seems to worry about fixing it.
Louis Gerbarg
· 2 months ago
If I can step in here for Ben for a second. People keep talking about HW acceleration as though using any OpenGL call means it is accelerated. Of course they are stamping a texture and putting out on the screen using some HW features. Its not like there is any way to access a physical linear framebuffer and directly blit to it.
The ARM11 isn't a crappy chip, it is a crappy chip for software compositing, and Ben claims that they will be able to move to HW compositing which gets that out of the critical path. But ultimately, the ARM11 is to blame from the crappy performance of the Flash renderer because it will have bad performance with basically any software renderer. Adobe is to blame for not having a HW renderer ready, but at some point people need to bite the bullet and ship. Don't get me wrong, I'm still skeptical they can pull it off in a generic sense that works well for lots of exisiting Flash code, but the way it works now makes sense from a project management perspective.
The issue is that the current apps aren't using the GPU to perform any of the compositing. Strictly speaking, that is what one would expect from a fairly new product, for 2 reasons. 1) In most cases you need to have a software renderer in any event because you might fall off the HW path depending on the features you are using. Just look at how an iPhone 3G performs when you use transparencies. 2) It is generally easier to port the software render to new architectures, so it is what is running first, then you write lots of platform specific code to get HW rendering working. This is the same progression as Quartz to Quartz Extreme, etc.
Also, the ram and software renderer are actually a huge issue. The GPU and CPU share access to the bus and don't necessarily have enough unchip resource to keep multiple frames around. That means that those software composites are reading and writing, causing tons of bus transactions, which then contend with the GPU trying to grab the texture out of ram. I would not be shocking if the frame rate is not bound by CPU, but memory stalls.
Ben Garney
· 2 months ago
Thanks, Louis - I appreciate your excellent write up. :) (See reply further down for my thoughts on this thread.)
Ben Garney
· 2 months ago
Didn't post this in my reply lower down but I agree on the CPU thing. Desktop has ~1mb or better cache with prefetch. iPhone has 256kb cache at best, and 16kb at worst, and unclear what prefetch strategy it has. In addition, there is less and slower RAM. Rough estimate puts the iPhone RAM at something like 1/10th the speed of desktop RAM.
Software rasterization has to have big fast cache and fast RAM to be manageable (that's one big reason why GPU is fast), so even though on MHz it looks like about a 5x gap between desktop 3ghz chip and iPhone CPU, it can actually be much worse depending on workload.
I know the haters will interpret this as "omg ben is blaming iphone for his performance woes," but they have bad reading comprehension skills. There is a real, measurable difference between desktop and iPhone hardware, and you have to be very careful to get solid performance out of the iPhone.
Emanuele Cipolloni
· 2 months ago
I believe most of people reading here comprehend enough the distinction between a portable device and a desktop computer, at least at size level :-) Given that, I understand that someone can define "crappy" the performance of chip only if he/she is comparing a particular device to other devices in the same field rather than to a desktop as it would not make any sense (like saying the beetle has a crappy engine compared to a Ferrari, true, but you're comparing a 10K car to 500K car.....)
There is at least one company producing a compatible embedded Flash Player using the tessellation method to offload rendering completely to the hardware and even solving the FSAAA problem thanks to it. Their stuff is used in many softwares, honestly I didn't hear anybody laughing about them.
Since GLUT is not available on OpenGLES, we can only talk about custom tessellation routines.
Emanuele Cipolloni
· 2 months ago
Yes, I understand completely what you say. But there is a play on words here, by "HW accelerated" they really mean "not updating the textures so frequently", nothing is really accelerated, they really use the OpenGLES for its ability to blit faster to the screen and nothing else (not completely, the 2.5D stuff actually uses the 3D features but that's a digression.....). This would be equivalent to pre-saving all sprites to PNG from Flash IDE and then load them up as textures (pretty much what we do now) losing anyway the dynamic nature nature of Flash shapes and sprites.
Instead of rushing to release another version, they really should take an hard look at the code base of the Flash Player in general and start to create several rendering backends, rather just relying on the software one all the time.
They start to make use of hardware video decoder, but again that's just video, the vector rasterizer is still software; they made use of multicore chip, dividing horizontally the frame buffer by the number of available cores and assign to each core a strip to rasterize. Yes, it is accelerated, by taxing completely the CPU now (look at Mac) and this sounds more like the proverbial use of a cannon to kill mosquitos without really solving the issue at all.
On the device front, the things are still pretty messed up, I agree that from a project management point of view that was the only (maybe?) option they had, but if the result is so average why bother?
To efficiently render vector shapes, OpenGL offers some nice features on desktop systems, like the ability to upload polygons and ask to tessellate then texture them directly to the HW, if portable systems could offer similar options it would be almost trivial for Adobe to offer a completely HW based rasterization backend. OpenGLES do not offer such tessellation facility, that means they would have to do it in software (and it is not a trivial thing to do), then upload the resulting triangles strips, probably this will make the things even slower compared to now. Without considering that OpenGLES 1.1 do not even support antialiasing so rasterized polygons would look just horrible.
More than all the problems above, what surprises me is the closeness of the resulting conversion, I mean, if you know that you cannot target the Flash Player for the iPhone, why just don't take the smart route and allow people to tweak your converted code?
I have the impression they are selling the illusion to a lot of Flash developer that just pressing a button will do the magic, but targeting Flash Player 10 on 2.5GHz with 4GB RAM is certainly different from a small device.
And this whole strategy of blaming on the device when it is slow because it cannot run properly a resource hog runtime is plain silly and annoying at best.
Ben Garney
· 2 months ago
Actually, they are looking at multiple hardware backends - they certainly have them for the other 19 mobile platforms they target. You're right that the current HW support on iPhone is simply better compositing, but that goes a long way - and in fact I, personally, don't even use the vector renderer on the _desktop_, so it's all I need. Most ObjC 3d apps use HW as a fast blitter, too - look at Popcap's games for instance.
Using the GL tesselator in any serious application is like showing up to a drag race in a VW bug - you're gonna get laughed off the track. One of the big reasons Flash is popular with artists is that it offers 100% consistent quality/appearance across platforms. They don't care that much about speed; they want it to accurately reflect their design down to the pixel. My sense is that if Flash violated that promise, they would lose massive market share amongst artists.
So they are stuck (to a certain degree) with the software rasterizer. On higher end devices (with better pixel shader support) it is possible to match the software renderer, so as 3d capabilities get better you'll see more of that. But they have to cater to the low-end users to keep their market share, so only when the lowest end users have great penetration with that hardware can they switch wholesale to HW support. If they do it too soon they risk splitting their userbase and making content creation a huge pain.
I don't think anyone is blaming the device for performance unduly. Look - bottom line, it is a slower device than a PC. Even with hardware acceleration. So getting Flash to run smoothly on it is like putting the proverbial ten pounds of crap in a five pound bag. The same applies for every mobile and set top device. If you want to just do a port that is mediocre, you can do that with little work using the new compiler; if you want to make it good, it's more work (but still a lot less work than porting to ObjC). It is 100% possible to build great content on the iPhone, with Flash or with ObjC or with MonoTouch or Python or iTGB or many other technologies.
Jensa
· 2 months ago
To me, just pressing a button and getting a working iPhone app is pretty much magic? Why not trust Adobe's skilled engineers to make something worthwhile rather than bashing Alpha quality software? ;-)
I honestly have no illusions about final speed but Flash Authoring targeting other platforms than SWF is a very interesting development! iPhone authoring from a PC is cool by itself, isn't it?
J
BBMedia
· 2 months ago
I've always enjoyed reading your write ups Ben... even though tons of it blows right over my head. But hey, this is the way we learn. Sit at the table with people who know more than you. This appears to provide an awesome toolset and pipeline for getting game ideas onto the iPhone. Looking forward to more of the stuff you guys are pumping out in the "Lab"
-Benjamin Bradley
Ben Garney
· 2 months ago
Hey Ben - awesome to know you're a reader. :) All this iPhone stuff is in Flash CS5, so when it's out you can be your very own iPhone developer. :)
BBMedia
· 2 months ago
I won't have to wait for CS5, since I've been working with the iPhone SDK for the past few months and I'm aiming to release my first iPhone app in November. Not bad for a bizdev/marketing guy. :)
And me not waiting for CS5 is probably a good thing for all iPhone owners, I'm sure they don't want me releasing "80's Night with Pat" again.
chris
· 2 months ago
i've played native iphone games and they run very well. no lagging, brief load times, few crashes. 'trading stuff' seems extremely laggy.
i am a flash developer myself, but i keep it on the desktop. i really hope apple continues disregarding flash for the iphone until (and if) adobe fixes its massive performance issues, which are entirely their fault.
Ben Garney
· 2 months ago
One of my main points was that the performance issues are Adobe's fault in the sense that at the time I needed to put the app on the store for them to demo it, the hw support wasn't in. I am waiting for Adobe's SDK to stabilize a bit before I ship, but I have a version of Trading Stuff running on my phone right now that runs great.
The iPhone hardware is great. It's a fantastic phone, with good GPU, decent CPU, and great build quality. I'm very happy Adobe is taking advantage of its capabilities, and I want to stress it is entirely possible to build good apps with their tech. I'm sorry you dislike Adobe's UI decisions, but I don't think that that reflects on the quality of their Flash port.
Physics Games
· 2 months ago
good post
Matt Rix
· 2 months ago
This is by far the best blog post and discussion on CS5's Flash>iPhone that I've read yet.
Ben Garney
· 2 months ago
Thanks, Matt! (For the compliment and the tweet. :)
Jeremy Alessi
· 2 months ago
"the iPhone is something like a tenth of the machine your desktop PC is. So if you want to do incredible 3d graphics and hand-optimize your assembly, maybe you should look into using XCode and Objective C, and build your content and code explicitly for the iPhone."
Or you could just use Unity ;) No BS, I've got a project running an 8,000 poly boned character (duck) with ragdoll physics running nicely on iPod Touch 2G and above. Granted iPhone 3G and below hardware runs a bit slow but it's still very playable. The iPhone hardware is excellent, especially for game studios with a content creation bottleneck which wouldn't allow for any more detailed games even with powerful desktop hardware.
Otherwise, great article on early Flash iPhone development. I checked Trading Stuff out and thought it was pretty neat, especially for an 8 day project. I'm looking forward to seeing The Incredible Machine and Grunts:Skirmish on the platform.
Ben Garney
· 2 months ago
Sure, that is also an option. :) You will notice that their actual renderer is running in C on the device, though, so I don't think I was being too misleading. ;)
Cool game. Kind of gorey. :P
We'll see what comes out on the iPhone. We're more web oriented than device oriented, but we are always looking for cool projects. :)
Fahim Farook
· 2 months ago
Ben, just to digress slightly from the more technical side of things (and because the support URL for "Trading Stuff" gives me a DNS error ...) are you aware of a bug in "Trading Stuff" where after playing for a while you're unable to buy any more trade goods? The buy side gets this really long red line which extends to the left edge of the screen and you can't buy anything at all at that point. I've never been able to complete a game yet because of this issue ...
James Urquhart
· 2 months ago
Great write-up. It's nice to see flash finally appearing on the iPhone, albeit in app-form rather than in a fancy plugin. :)
A pity i haven't really taken flash seriously as a game development platform before this.
Performance-wise, getting into the "this is an embedded device with slow hardware" mindset does take a bit of getting used to. Nowadays i just refer to my little list of things which i know is slow. e.g. uncompressed texture uploads, disk i/o, various system api's, etc.
chris
· 2 months ago
I'm glad to see any article based on actual experience developing Flash for the iPhone. I'd like to see more detail though. Maybe after you finish optimizing for the GPU you could write up some specifics of what you learn along the way?
All Comments
However, I don't get why a bad performance of the actual Flash software renderer is to blame on the "crappy ARM11" chip or the "very slow ram", iPhone is not designed and sold with the "Made For Flash" sticker, Apple give plenty of resources (may I add free?) to develop properly on the iPhone and it is not really necessary to get down to ARM assembler in order get smooth 3D games at high frame rates or any other application.
It would be worth noting also that the converted Flash app is always HW accelerated as it uses the 3D chip to stamp texture on the screen but it actually uses it in the wrong way. When you use the "cache AsSurface" flag on a sprite you're not activating any special hardware setting in the iPhone side, you're only telling to the Flash soft-renderer to consider an already rendered object as static, in this way, the texture associated to a certain sprite is not rendered again, but for this to work, most if not all display list objects have to be static or very small to avoid the bottleneck in texture transfer from ram to 3D chip.
Nobody doubt that Adobe teams are full of very talented people, just not sure how focused they are, I bought my CS4 over 1 year ago and I feel kind of frustrated that they jump and down to show CS5 when this stuff that costed me over 2000 euros still crash like it is a beta and nobody seems to worry about fixing it.
The ARM11 isn't a crappy chip, it is a crappy chip for software compositing, and Ben claims that they will be able to move to HW compositing which gets that out of the critical path. But ultimately, the ARM11 is to blame from the crappy performance of the Flash renderer because it will have bad performance with basically any software renderer. Adobe is to blame for not having a HW renderer ready, but at some point people need to bite the bullet and ship. Don't get me wrong, I'm still skeptical they can pull it off in a generic sense that works well for lots of exisiting Flash code, but the way it works now makes sense from a project management perspective.
The issue is that the current apps aren't using the GPU to perform any of the compositing. Strictly speaking, that is what one would expect from a fairly new product, for 2 reasons. 1) In most cases you need to have a software renderer in any event because you might fall off the HW path depending on the features you are using. Just look at how an iPhone 3G performs when you use transparencies. 2) It is generally easier to port the software render to new architectures, so it is what is running first, then you write lots of platform specific code to get HW rendering working. This is the same progression as Quartz to Quartz Extreme, etc.
Also, the ram and software renderer are actually a huge issue. The GPU and CPU share access to the bus and don't necessarily have enough unchip resource to keep multiple frames around. That means that those software composites are reading and writing, causing tons of bus transactions, which then contend with the GPU trying to grab the texture out of ram. I would not be shocking if the frame rate is not bound by CPU, but memory stalls.
Software rasterization has to have big fast cache and fast RAM to be manageable (that's one big reason why GPU is fast), so even though on MHz it looks like about a 5x gap between desktop 3ghz chip and iPhone CPU, it can actually be much worse depending on workload.
I know the haters will interpret this as "omg ben is blaming iphone for his performance woes," but they have bad reading comprehension skills. There is a real, measurable difference between desktop and iPhone hardware, and you have to be very careful to get solid performance out of the iPhone.
There is at least one company producing a compatible embedded Flash Player using the tessellation method to offload rendering completely to the hardware and even solving the FSAAA problem thanks to it. Their stuff is used in many softwares, honestly I didn't hear anybody laughing about them.
Instead of rushing to release another version, they really should take an hard look at the code base of the Flash Player in general and start to create several rendering backends, rather just relying on the software one all the time.
They start to make use of hardware video decoder, but again that's just video, the vector rasterizer is still software; they made use of multicore chip, dividing horizontally the frame buffer by the number of available cores and assign to each core a strip to rasterize. Yes, it is accelerated, by taxing completely the CPU now (look at Mac) and this sounds more like the proverbial use of a cannon to kill mosquitos without really solving the issue at all.
On the device front, the things are still pretty messed up, I agree that from a project management point of view that was the only (maybe?) option they had, but if the result is so average why bother?
To efficiently render vector shapes, OpenGL offers some nice features on desktop systems, like the ability to upload polygons and ask to tessellate then texture them directly to the HW, if portable systems could offer similar options it would be almost trivial for Adobe to offer a completely HW based rasterization backend. OpenGLES do not offer such tessellation facility, that means they would have to do it in software (and it is not a trivial thing to do), then upload the resulting triangles strips, probably this will make the things even slower compared to now. Without considering that OpenGLES 1.1 do not even support antialiasing so rasterized polygons would look just horrible.
More than all the problems above, what surprises me is the closeness of the resulting conversion, I mean, if you know that you cannot target the Flash Player for the iPhone, why just don't take the smart route and allow people to tweak your converted code?
I have the impression they are selling the illusion to a lot of Flash developer that just pressing a button will do the magic, but targeting Flash Player 10 on 2.5GHz with 4GB RAM is certainly different from a small device.
And this whole strategy of blaming on the device when it is slow because it cannot run properly a resource hog runtime is plain silly and annoying at best.
Using the GL tesselator in any serious application is like showing up to a drag race in a VW bug - you're gonna get laughed off the track. One of the big reasons Flash is popular with artists is that it offers 100% consistent quality/appearance across platforms. They don't care that much about speed; they want it to accurately reflect their design down to the pixel. My sense is that if Flash violated that promise, they would lose massive market share amongst artists.
So they are stuck (to a certain degree) with the software rasterizer. On higher end devices (with better pixel shader support) it is possible to match the software renderer, so as 3d capabilities get better you'll see more of that. But they have to cater to the low-end users to keep their market share, so only when the lowest end users have great penetration with that hardware can they switch wholesale to HW support. If they do it too soon they risk splitting their userbase and making content creation a huge pain.
I don't think anyone is blaming the device for performance unduly. Look - bottom line, it is a slower device than a PC. Even with hardware acceleration. So getting Flash to run smoothly on it is like putting the proverbial ten pounds of crap in a five pound bag. The same applies for every mobile and set top device. If you want to just do a port that is mediocre, you can do that with little work using the new compiler; if you want to make it good, it's more work (but still a lot less work than porting to ObjC). It is 100% possible to build great content on the iPhone, with Flash or with ObjC or with MonoTouch or Python or iTGB or many other technologies.
I honestly have no illusions about final speed but Flash Authoring targeting other platforms than SWF is a very interesting development! iPhone authoring from a PC is cool by itself, isn't it?
J
-Benjamin Bradley
And me not waiting for CS5 is probably a good thing for all iPhone owners, I'm sure they don't want me releasing "80's Night with Pat" again.
your blame of the iphone's hardware sounds exactly like the excuse an adobe developer was using in the adobe forums. he claimed cs4 did not work properly with exposé (window switcher) in os x because apple didn't support their software. the thing is that cs4 apps were the only ones with problems and just so happen to be the ones not following apple's design guidelines. cs4 is a sloppy mess of 'fake' windows floating on top of an actual os x window.
so just as the adobe developer blamed apple for their own poor design choices, you are blaming apple for not supporting flash's clearly unsupported platform. flash needs so many resources for such simple tasks. what you and adobe are doing are creating a very tall truck and are now asking cities to rebuild their bridges taller to support your truck which should be shorter in the first place.
i am a flash developer myself, but i keep it on the desktop. i really hope apple continues disregarding flash for the iphone until (and if) adobe fixes its massive performance issues, which are entirely their fault.
The iPhone hardware is great. It's a fantastic phone, with good GPU, decent CPU, and great build quality. I'm very happy Adobe is taking advantage of its capabilities, and I want to stress it is entirely possible to build good apps with their tech. I'm sorry you dislike Adobe's UI decisions, but I don't think that that reflects on the quality of their Flash port.
Or you could just use Unity ;) No BS, I've got a project running an 8,000 poly boned character (duck) with ragdoll physics running nicely on iPod Touch 2G and above. Granted iPhone 3G and below hardware runs a bit slow but it's still very playable. The iPhone hardware is excellent, especially for game studios with a content creation bottleneck which wouldn't allow for any more detailed games even with powerful desktop hardware.
http://www.slidetoplay.com/video/ragduck-hunt
Otherwise, great article on early Flash iPhone development. I checked Trading Stuff out and thought it was pretty neat, especially for an 8 day project. I'm looking forward to seeing The Incredible Machine and Grunts:Skirmish on the platform.
Cool game. Kind of gorey. :P
We'll see what comes out on the iPhone. We're more web oriented than device oriented, but we are always looking for cool projects. :)
A pity i haven't really taken flash seriously as a game development platform before this.
Performance-wise, getting into the "this is an embedded device with slow hardware" mindset does take a bit of getting used to. Nowadays i just refer to my little list of things which i know is slow. e.g. uncompressed texture uploads, disk i/o, various system api's, etc.