Goodbye Google App Engine (GAE)

This is a post rewrite. The original post can found at the bottom of this post.

The reason why I am rewriting this post is because Patrick Chanezon (from Google), has added a kind and respectful comment to this post. Given the huge amount of traffic this post has generated (never expected nor wanted) I don't want to damage the GAE project which can be a great platform in the future. People haven't understand  my point apparently because of the dramatic writing style of the original post.

The reason why we left the GAE platform is solely because of its unreliability as of september-octuber-november 2010. The last 3 months we saw the site down more times that we considered reasonable and we couldn't find a way to avoid it so eventually we decided to abandon the platform. When the GAE team be able to stabilize the platform, it will perhaps be a great platform for scalability.

I emphasized the GAE limitations because some of them were hard to work around when we were developing although we did a great job making everything work, but when the platform let me down because of its many 500 errors,I got the feeling that we had been loosing our time with a platform not enough mature.

As a young and small entrepreneur (I am the one who fund our projects training developers all around the country), the development effort was big and I calculated the amount of money and wrote it in the post, but any experienced developer knows that the quantity I wrote is not that much. Some of my readers are friends who have invested money in the project and I wanted to give them an idea of the effort we made in terms of money in the GAE platform.

When we started developing on GAE one year ago, there was less documentation that there are today. But even today, I haven't found any document saying... "if the application can't load into memory within 1 second, it might not load and return 500 error codes". This is something that should be written in the documentation as soon as possible to help others understand if GAE is a good fit for their apps or not. Django can load into memory in less than a second in my local machine I guess but apparently not in GAE and we build our app in Django because Webapp framework miss many things like security. With Django we don't have to care about XSS and i18n is easy while in Webapp it not that easy. So if GAE can't run with Django we had to migrate. But we were using Django Helper, developed but Google engineers if I recall correctly so it is quite disappointing to read from another Google engineer that now we have to load under a second. Sounds contradictory. Sounds not professional.

Regarding the GAE for Business, I believe this is really new. I remember joining the business plan/account when it was kind of beta and I thought it was still in beta because I never got news about it. So yes, we wanted to pay for better support and SLA and at that point we couldn't. As a developer I can't be checking this things every day because I've got many things to do so I guess Google have to inform me if the business account is now working or what.
The support from GAE engineers have been really bad and this is a mistake they know but I hope in the future it gets better for those who are investing in the platform.

In our current hosting we still don't do JOINs because we don't need them and we've learned a lot of things about scalability thanks to GAE restrictions. Data DEnormalization is one of them and we keep it like that. I wrote about the limitations because wanted to warn users who haven't  read about No-SQL. When I said "normalization" in the original post I meant that before saving strings, we transform them in a way that we can search for them (uppercase and things like that). When the application was running, it wasn't slow. We managed to optimize our algorithms and developed a caching architecture which works really well and have a nice API.

Although many people thought we have lost money for not reading docs, the reality is that we left GAE for its unreliability and nothing more.

We enjoyed certain things from the GAE platform like the feature of changing versions with a single click and the ability to add new fields to Kinds dynamically.

It is nice if GAE success and I am happy to see users who are enjoying the platform but for us, I have had enough for now. I also don't want to try Azure. The more experience I gain, the less I trust platforms/frameworks which code I can't see.

Thanks for reading and understanding my English which I think should be understandable.

====================================================

--------- Original post written on 21 nov 2010:

Choosing GAE as the platform four our project is a mistake which cost I estimate in about 15000€. Considering it's been my money, it is a "bit" painful.

GAE is not exactly comparable to Amazon, Rackspace or any of this hosting services, because it is a PaaS (Platform as a Service) rather than just a cluster of machines. This means that you just use the platform and gain scalability, high availability and all those things we want for our websites, without any other software architecture. Cool, isn't it.
You do not pay until you get a lot of traffic to your site so it sounds very appealing for a small startup. I read about this in a book called "The web success startup guide" by Bob Walsh.
So yes, everything looked cool and we decided to go with this platform a couple of months ago.
It supports Python and Django (without the ORM) which we love so we tried it out. We made a spike, kind of 'hello world' and it was easy and nice to deploy.

When we started developing features we started realizing the hard limitations imposed but the platform:

1. It requires Python 2.5, which is really old. Using Ubuntu that means that you need a virtualenv or chroot with a separate environment in order to work with the SDK properly: Ok, just a small frustration.

2. You can't use HTTPS with your own domain (naked domain as they called) so secure connections should go though yourname.appspot.com: This just sucks.

3. No request can take more than 30 secons to run, otherwise it is stopped: Oh my god, this has been a pain in the ass all the time. When we were uploading data to the database (called datastore a no-sql engine) the uplaod was broken after 30 seconds so we have to split the files and do all kind of difficult stuff to manage the situation. Running background tasks (cron) have to be very very well engineered too, because the same rule applies. There are many many tasks that need to take more than 30 secons in website administration operations. Can you imagine?

4. Every GET or POST from the server to other site, is aborted if it has not finished within 5 seconds. You can configure it to wait till 10 seconds max. This makes impossible to work with Twitter and Facebook many times so you need intermediate servers. Again, this duplicates the time you need to accomplish what seemed to be a simple task

5. You can't use python libraries that are build on C, just libraries written in python: Forget about those great libraries you wanted to use.

6. There are no "LIKE" operator for queries. There is a hack that allows you to make a kind of "starts with" operator. So forget about text search in database. We had to work around this normalizing and filtering data in code which took as 4 times the estimated time for many features.

7. You can't join two tables. Forget about "SELECT table1, table2..." you just can't. Now try to think about the complexity this introduces in your source code in order to make queries.

8. Database is really slow. You have to read about how to separate tables using inheritance so that you can search in a table, get the key and then obtain its parent in order to avoid deserialization performance and all kind and weird things.

9. Database behavior is not the same in the local development server than in google servers. So you need some manual or selenium tests to run them against the cloud, because your unit and integration tests won't fail locally. Again this means more money.

10. If you need to query on a table asking for several fields, you need to create indexes. If those fields are big, it is easy to get the "Too many indexes" runtime exception. That will not happend to you in a "hello world" application, but enterprise applications that need to query using several parameters are easy to run into this trouble. If you use a StringListProperty, you might not find this problem until one user populates this field with a long list. Really nasty thing. We had to re-engineer our search engine once in production becase we didn't know this till it happened.

11. No query can retrieve more than 1000 records. If there are more than 1000, you have to deal with offsets in your code to paginate results. Taking into account that you have to filter results in code sometimes because of the the limitations of GQL (the query language), this means again more complexity for your code. More and more problems, yeiiii!

11. You can't access the file system. Forget about saving uploads to filesystem and thouse tipical things. You have to read and learn about the inconvenient blobs. Again, more development time wasted.

12. Datastore and memcache can fail sometimes. You have to defend your app against database failues. What? Yep, make sure you know how to save user data when this happens, at any time in the request process, but remember... you can't use the filesystem. Isn't it fun?

13. Memcache values max size is 1megabyte. Wanted to cache everything? You were just dreaming mate, just 1 megabyte. You need your own software architecture to deal with caching.

There are some more things that make the development hard with this platform but I am starting forgeting about them. Which is great.

So why did we stay with this platform? Because we were finding the problems as we go, we didn't know many of them. And still, once you overcome all the limitations with your complex code, you are suppoused to gain scalabilty for millions of users. After all, you are hosted by Google. This is the last big lie.

Since the last update they did in september 2010, we starting facing random 500 error codes that some days got the site down 60% of the time. 6 times out of 10, users visiting the site couln't register or use the site. Many people complained about the same in the forums while google engineers gave us no answer. After days they used to send and email saying that they were aware of the problem and were working on it, but no solutions where given. In november, an engineer answered in a forum that our website had to load into memory in less than 1 second, as pretty much every request was loading a completelly new instance of the site. 1 second? Totally crazy. We have 15000 lines of code and we have to load django. How can we make it under 1 second?
They encourage people to use the light webapp framework. Webapp is great for a "hello world" but it is bullshit for real applications.
So once we suffered this poor support from google and we understood that they were recommending GAE just for "hello world" applications, we decided to give up on it finally.
This is not serious nor professional from google. We are more than dissapointed with GAE.

For us, GAE has been a failure like Wave or Buzz were but this time, we have paid it with our money. I've been too stubborn just because this great company was behind the platform but I've learned an important lesson: good companies make mistakes too. I didn't do enough spikes before developing actual features. I should have performed more proofs of concept before investing so much money. I was blind.

After this we wanted to control the systems so we have gone to a classical hosting based on PostgreSQL, Nginx, Apache2 and all that stuff. We relay on great system administrators who give us great support and make us feel confident. We are really happy to be able to make SQL queries and dump databases and all those great tools that we were missing in the dammed GAE.

Because we are software craftmen, we designed all our code using TDD and the rest of XP practices. Thanks to this, we migrated 15000 lines of code in just 1 week (and just a pair of developers) from one platform to another which has been a deep change. From No-SQL back to SQL. From one database access API to another. If we didn't had these great test bateries, migration would have taken months.
Migrating data took us 1 week because of encoding problems and data transformations.

You might agree or not with me on this post but there is one certain reality: developing on GAE introduced such a design complexity that working around it pushes us 5 months behind schedule. Now that we had developed tools and workarounds for all the problems we found in GAE, we were starting being fast with the development of features eventually, and at that point we found the cloud was totally unstable, doggy.

Enjoyed reading this post?
Subscribe to the RSS feed and have all new posts delivered straight to you.
  • Yaroukh

    @Markandey Singh: “2)There is no Session, you have to come up with your own.”
    wow, just wow
    of course, GAE supports sessions
    this pretty much sums majority of the GAE-bashings

  • Ed

    Very insightful post, thank you for sharing your real world experiences with the community.

    At my old company we built one of our flagship products on App Engine because 1) we thought the massive amounts of limitations and issues would go away in time because of “Google Innovation” and 2) we were building a Google Apps based product so there was a marketing and product management affinity, and 3) Google hyped up AppEngine so much we figured they’d have to make good on it and keep it live or have big egg on thier face

    Our developers went through many nightmarish development cycles to get basic functionality out – and these weren’t script kiddies who never saw a scalable app before. That said we finally got the app out to production, but the “Google Innovation” never came. Despite the 101-level Google I/O talks year after year, App Engine is still basically a sandbox mish-mash of various web technologies. When you look at how far AWS has come in the same time frame, you wonder how innovative Google really is outside of search.

    On the marketing and product management usefulness of AppEngine for Google Apps plugins, there really is no great advantage here. You would think you’d get some better request lifetimes call Apps API from GAE, but in many cases they are worse.

    On the third point, as we saw from Wave and I predict Buzz, Google has no problem pulling the plug on products that don’t catch on. GAE is getting it’s butt kicked by AWS and the metaclouds like EngineYard that run on it from one side, the real platforms like Saleforce AppExchange on the other, and the SMB focused function driven Saas platforms like Wufoo in the middle (yeah, Wufoo is a platform if AppEngine is).

    For these reasons, it won’t be long until GAE is either cut, crippled, or changed so fundamentally that any investment today would be wasted.

    We did make it out

  • anon

    Thank you thank you thank you Carlos Ble for posting this. You saved my ass.

  • http://hosting4developers.com Hosting 4 Developers

    The essense of the App Engine apologists’ argument is that App Engine is solely for high-traffic sites, and the AE limits are justified because AE provides high scalability.

    The problem with that argument is that, in reality, App Engine simply is not scalable, even if you know how to write good code for it. There have been numerous blog posts and comments on reddit.com/r/appengine about how AE is prone to random datastore errors, memcache errors, mail sending errors, etc. Those errors seem to happen all the time. How can we believe that AE is scalable when it can’t even handle low-traffic sites without random errors all of the time?

  • W GAE

    Yeah, newbie devs, please leave GAE to smart people, we don’t need your fat old code running on the same cloud that powers our super apps! W GAE!

    http://code.google.com/appengine/forum/?place=topic%2Fgoogle-appengine%2FoTEUi39W2Ks%2Fdiscussion
    http://www.reddit.com/r/programming/comments/e9n3l/goodbye_google_app_engine/?limit=500

  • Alec

    So, do you noted all this AFTER migrating your whole project?, Kind of joke blaming GAE for you loosing all that money. Maybe is true that GAE doesn’t fit your needs, but all that lost money is on you.

  • Alex

    Thanks for providing a point of view next time a new client asks about Google’s app engine. We had an absolutely horrible time with it for one client and have vowed never to go back. And our problems had NOTHING to do with the limitations and everything to do with random errors that popped up with no understandable cause. Said client was much happier when we recoded and moved everything over to Amazon’s cloud. Maybe our code sucks and we don’t know what we are doing, but then I am glad that Amazon’s cloud is powerful enough to overcome our limitations ;).

    Sounds to me like some of the trolls commenting in support of the platform are probably Google employees. Someone mentioned Google-hookers – lol. I’ll have to remember to use that one!

  • NOYB

    App Engine is not without its problems. Anyone who argues that it is has obviously never targeted it. The reason I am critical of the authors comments is because he obviously didn’t take the time to study the published documentation and learn about the platform he was targeting. He also didn’t keep up with releases which included new features and bug fixes. If he was limited by any single named value being larger than 1 meg in mem cache then he should have known 1) there is a limit; 2) how much data was actually being cached; 3) spead the values out among numerous named value pairs to overcome the limit.

    The author comes off as a big crybaby who never learned how to drive a car, gets in a car and crashes it and then blames the car manufacturer for his problems. Get real!

    The bottom line is that App Engine is beyond the reach of the casual or hobbyist developer (they should stick with the many hosting services that are available for a reasonable monthly fee which provide them with their own server and on which they can bog it down with inefficient processing as much as they want) and their musings only serve as a distraction to the real issues facing real developers who are trying to develop real scalable applications.

  • http://www.zehn.de/ stephan

    Our Website is running on App Engine since November 2008. We carefully studied the documentation and decided to stat on GAE. We liked the idea of scalabilty.

    After two years we still like the way to just scale up when it is needed.

    But two problems are really hard to solve:
    – Downtimes! Dude! Why so much? We still pay for the service! Server Error 500 – HATE!
    – The 1000 limit. Ouch! Try a simple search with that. It was a running joke here: Google App Engine doesn’t provide proper technology to execute search-queries. Bummer!

    So as a “pioneer” I wouldn’t recommed anybody to start on GAE. Just not ready…

  • Ronnyek

    I’ve been using GAE building a pretty good size web app… and I got to say… I think your reasoning and justification for throwing GAE away is just asinine. Do I think some of the restrictions with GAE are a bit of a pain to work around? Yes. But its ok to admit you just don’t like it.

    Is it the right solution for every problem? No. But complaints about the data store, normalization etc, are just side effects of being a no-sql’esque data storage mechanism. (if you were trying to build anything with nosql db, you’d be encountering many of the same problems).

    I’ve seen java and python “warmup” times take a hit from time to time, but arguments against memcache etc, sound like they may have been a bad implementation. After all cache is supposed to be lazily populated… cant GUARANTEE keys are going to be there… handle it accordingly.

    As for long running requests… you realize why http made it this far right? Maintaining a bunch of persistent TCP connections is a lot heftier than just take a request, quickly respond to it, and disconnect. With the exception of downloading/uploading, I’d make the argument if you are running a request longer than 30 seconds you are definitely doing something wrong.

    In closing, I don’t mean to bash the basher, just mean to say its just different than what most are used to. The restrictions are there as a result of technologies used under the scenes. They are also there to ensure your apps scale as much as possible. As for the performance, other than the occasional python warm-up time, I’ve not experienced any performance problems.

    Sorry for the rant, but I think maybe conceptually the appengine architecture is too much to grasp for some or just rubs them the wrong way, but my experience is that its not so much the platform itself, but rather the changes you have to make to traditional apps to make them make sense here.

  • Michel

    Thanks for the informative post OP, I had no idea GAE was so crippled. Pay no attention to the fanboys who seem to think you can’t sum your experiences on your own site because some of that info can be found elsewhere on the web.

  • Chris

    These flamers are way out of line here. This article is a good look into when not to use GAE. Thanks for the insight!

  • Phil.H.

    Thats pretty interesting. I actually had very similar feelings about GAE Java just a month or so ago and moved my project away from it. I later realized that Java was a part of the problem too and moved to Python. Amazingly with it being my first ever Python project I ported a couple months of Java work within 3 days. But then I decided to give GAE Python a shot and I must say I’ve been happy with what I’m seeing so far. I’m not in production…still developing and testing…so that could change. And maybe being new to Python I somehow skated around the 2.5/3.0 problem by accident 🙂

    It sounds though like your app does some heavy processing often and will not fit inside GAE. No biggie. I don’t see why people feel the need to flame. I don’t really believe you said that GAE sucks except for some issues here and there. I have had a couple of app ideas that probably wouldn’t fit with GAE because of their restrictions. But for the projects that fit within them its a very cost effective way to get something off the ground.

  • NOYB

    @Chris

    Calling it like it is does not make someone a flammer. If the author cannot take criticism for their words then perhaps they shouldn’t have publicly exposed their “insights” to public discourse and ridicule. This is true democracy and unless you are a communist you should welcome it. Labeling those of us who have participated in this public discourse as flamers is a veiled attempt of liberal leaning leftist mind control which has no place when it comes to blogging.

  • adam sah

    hi, I’m a longtime appengine developer and built several $1M+ products on it. Yes, it has limitations and yes it has outages.

    v1.4 (just announced) removes the worst of the limitations, and the “deadsnakes” project has a super easy port of python 2.5 for ubuntu, but it’s true: 2.5 is ancient and therefore has many limitations (e.g. internationalization).

    my overall 2c, having fought battles with ruby and other platforms: GAE is different and it helps to have spare brainiacs on staff, but it’s workable and nice. I like having (far) fewer components and no pager to carry.

    finally, it’s *very* unlikely that GAE will be cut– Google depends on it for numerous services and some of their largest customers use it. It seems like Google Apps is depending on GAE more and more.

    adam
    former google engineer (invented Google Gadgets)

  • NOYB

    @Phil H.

    Have you heard of TASKS for ‘heavy’ and long running processes??? If you haven’t then that’s a shame because guess what? They really do provide the ability to get a lot of work done in the background. The problem is though that you have to learn how to use them and that requires reading the documentation and asking questions on the App Engine group – services everyone is able to freely take advantage of if one is smart enough to avail themselves to doing so… something the author of this dribble obviously didn’t do.

  • http://pbjbreaktime.com/ Icechen1

    Wow, that’s pretty lacking of Google.

  • http://faisaliqbal.net Faisal Iqbal

    Good and informative Post! though I am not agree that GAE is bad choice for every large scale application. I have work experience`with force.com that has even more such limitations.

  • Yaroukh

    @Michel: so your knowledge of GAE is basically “Google made it” yet you are brave enough to decide that people who tore apart the author’s complaints (because they know the platform well) must be fanboys 🙂
    aren’t you sore that Google does not support PHP on GAE? 🙂

  • http://alex.cloudware.it alex

    Couldn’t agree more with gimenete.
    RTFM, it’s all there.

    Just saw this tweet: http://twitter.com/chanezon/statuses/6765733581168642

  • stevko

    As an independent developer (and not any kind of fanboy), I can certainly say that my dev experience with GAE/J has certainly been a tough learning curve. Having just launch a very public website, I can also say that it scales to thousands of users very well.

    The hardest part is breaking all those ancient system architecture habits and adopting new patterns. For example, designing a stateless sessionless server is the way to horizontal scalability. Creating a data model out of objects rather than relational tables is the next generation of dbms. Writing code that gracefully handles every kind of error from http 500 to the database concurrency exceptions is the way to a bulletproof user experience.

    Every constraint GAE places on the application makes sense for a platform that can scale to 1000s of servers and millions of clients. There is no single file system that can handle that. No single database server has enough power to handle that. Long duration processes & outbound http requests lock up resources and directly degrade the user experience.

    The secret to building in the cloud is not to try to create a lake but rather make millions of rain drops.

  • Pingback: Goodbye Google App Engine | Large Data Matters()

  • http://undefinedvalue.com/ Kristopher Johnson

    Thank you very much for sharing your story. Don’t take the bashing personally. Your willingness to share what you’ve learned is a lot more valuable than the comments of all the idiots who just want to call you an idiot.

  • Pingback: links for 2010-11-22 « that dismal science()

  • http://carlosble.com Carlos Ble

    Hi guys,
    Thanks for your comments.

    I’ve posted a new entry as a reply:

    http://www.carlosble.com/?p=722

  • Pingback: » links for 2010-11-22 (Dhananjay Nene)()

  • Kamran

    Some points are valid while other’s are clearly stated by the GAE documentation.We have to choose what to use on bases of what is our need.It is correct that GAE suck on a lot of points.But moreover there rate of removing the obstacle is too slow compare the Windows azure which slowly but surely grabbing market and above all satisfying there customer on the promised functionalities.

  • Michel

    @Yaroukh

    No, I am well aware of what GAE is, but I always assumed it was pretty advanced and complete. Now, thanks to this post, I also know is that is severely limited and regularly goes down.

    If you are not willing to concede these facts and/or get your panties all wrapped up about someone pointing them out on his own site, you’re a fanboy. And if you are impolite and talk in a nasty tone of voice (like many of you do), you’re a flamer.

  • http://vitriolix.com vitriolix

    Thanks for posting this, always good to see what pains people hit with new technologies.

    I’m going to have to mostly side with commenters like @gimenete though. most of these bullet points are a basic misunderstanding the product, not a flaw in it. Sounds like you want GAE to be AWS, but it isn’t and that is what makes it interesting and highly scalable.

    The stuff about no file system access, no c libs, no joins, 1000 row limit on data store, 30ms timeouts… all of that is stated up front in the tutorial. These are design trade-offs that they state up front and that you should have known about in your evaluation of the product.

    That said, the stability issues are scary though, and important to hear first hand account of.

    thanks very much for sharing your experience, it is very valuable.

  • http://www.josht.com JT

    I’m still surprised at the people who are freaking out over the 1000 result limit, for one it’s stated in the docs, secondly, as I mentioned above, in MOST (notice I didn’t say “all”) situations it’s not needed… and where it is, you can usually work around it.

  • http://www.josht.com JT

    That being said, the limit was lifted a couple months back, but probably after you abandoned the platform.

  • NOYB

    @Michel said “No, I am well aware of what GAE is, but I always assumed…”

    Now Micahael, you know what they say about people who AssUMe, don’t you?

    But more amazingly, Michael, right after you admit that you really don’t know anything about GAE you have the cojones to say “If you are not willing to concede these facts…”.

    Now Michael, you must be losing your memory because just a moment ago you said that you were assuming what GAE is which is admitting that you have no idea what GAE is because you’ve never coded one line of code for it, now have you? With that little bit of deductive logic out of the way lets continue…

    Having no practical real knowledge of GAE yourself you’ve somehow managed to read the dribble produced by the author of this article, have taken everything he said as a given and are now calling anyone and everyone who has disagreed with him either 1) a fanboy, 2) a flamer or 3) a fanboy/flamer with their panties in a knot.

    May I suggest to you, Michael, that after having analyzed you that perhaps the only panties in a knot are your own and that they must be tied around your windpipe severely cutting off the oxygen to your peanut sized brain and hence short circuiting your mental process (like you ever had any in the 1st place).

    Please, Michael, I beg you to tell me that you have never coded in your life and, more importantly, that your swear from this day forward that you never will.

    Tata, Michael! Or is that tartar?

  • Pingback: NoSQL Daily – Tue Nov 23 › PHP App Engine()

  • http://razenha.wordpress.com Rubem Azenha

    Well, I guess that’s what happens when you bet all your money in a closed environment. Too bad you have to learn this by the hard way, but some of this restrictions were clearly documented (specially the request time restriction).

    I think none of the projects I worked in my entire career would fit in GAE restrictions. I really think that any serious project would have a huge overload of work to dodge GAE restrictions and I find hard to believe that any Google application can run in this environment.

    We all need to learn the lesson, don’t rely on closed environments. Because GAE may use FOSS tools like Python, but is not and environment you can really reproduce in your own server and you don’t really have access to it’s underlaying infra-estructure.

    Even Microsoft Azure seen to be more open than GAE. While it’s rely on proprietary technology, we all know exactly what’s the underlaying infra-estructure and can reproduce it in our own server. It’s easy do migrate a MS Azure app to another server. We can’t say the same about GAE… I can only imagine you must had a hell of a time migrating the code and the data…

    Luckily for the Ruby community, they have at least two PaaS providers that’s offer a open environment: Heroku and Engine Yard. I do not know if the Python developers have the same options. I guess the fact that Google itself provides a PaaS hosting makes it kind a hard to compete…

  • http://www.donhopkins.com Don Hopkins

    gimenete, you’re not just an asshole, but also an apologist. While you’re at it, let’s hear your rap on how PHP is really a very well designed language, and it’s the programmer’s fault for anything that goes wrong. And while you’re at it, please explain to us how it’s the deaf children’s fault for being molested, not the Catholic church’s or Pope Ratzinger’s fault for protecting the priests.

  • Steve

    While it’s true that the original poster admitted his mistakes in choosing the platform, it’s unfortunate that the narrative in this post couldn’t have been closer to “It didn’t suit this project and this is why…..”, though that information is still clear.

    In regards to publication of GAE ‘limitations’, I’m far from the most judicious reader of documentation and specs but I remember when first checking out GAE I was immediately made aware of those all of those limitations, as well as the reasons behind them. Furthermore, with just a little of my own digging I found endless examples, by the GAE team and other developers, of how to negate each of those design differences by designing an application the ‘GAE way’.
    It seems like a very fair and necessary trade off considering what you get back is pure scalability, which for small teams and projects without massive budgets is something they can often never achieve.

    I think it shows a complete lack of responsibility that you would try to assign any of the blame to GAE when, by your own admission, you were the one that made those very serious errors.

    As for the stability issues, which I would say is your only legitimate beef, those are certainly relevant, but I think it’s unrealistic to suggest that other services/solutions could compete with GAE when it comes to overall value especially when you factor in the ability to scale, the ease of admin/develpoment/deployment, and the general cost-efficient nature of the platform.

    Let’s also remember that any reliability issues are certainly due to GAE being an enormous project still under furious development, and shouldn’t be thought of as permanent problems.

    All in all I think posts of this nature aren’t serving anyone, and indeed you’ve created a lot of unwarranted bad publicity around a service which for many developers is, by an order of magnitude, a better solution than any other.

  • shreedinesh

    Thanks for the information on GAE. It’s really helpful to make the right decision on when to go for GAE and when not

  • nmatrix9

    Carlos thanks for pointing these issues out. I have read the GAE documentation, some of these issues I know about but some of the others such as the GET and POST timeout issues are new to me. Thanks for the warning now I’m definitely going to be checking out Amazon EC2.

  • http://www.inphina.com Vikas Hazrati

    We migrated a hugely successful enterprise timesheet and invoicing application to GAE a couple of months back. Did we face the above issues? Sure we did, did we find workarounds sure we did.
    Did GAE help us, you bet it did.

    We should understand that GAE is a platform. It is not a piece of hardware on which you would be able to put your OS, software and it works as per your needs. It is a PLATFORM which has to support thousands of application. Now when you are on a platform which is shared there are restrictions. It is not your backyard any more. There are constraints and most of these constraints would lead to innovation and a better design. If you look at the 30 sec constraint, it is a decent one. The key lies in making your logic and application work around that constraint because then you eventually arrive at a better design.

    We blogged about all our findings and our journey of migrating the application based on Spring, Wicket and Hibernate here http://thoughts.inphina.com/tag/gae/

    Also our take on GAE and Amazon EC2
    http://thoughts.inphina.com/2010/11/01/comparing-google-app-engine-and-amazon-ec2-on-technology/

  • Alex

    Carlos,

    Thanks for sharing all of your experiences. Normally, I don’t really comment on posts, but the bashing that’s going on by some people is unfair. You were brave enough to share your experiences and you do deserve credit for that and while some of you poins were design issues, you are completely correct with your comment “Can you predict the future?”. If your app was working and suddenly it doesn’t anymore, it is Google’s fault. While you might not have an SLA with Google, it’s great that you point out to others what might happen when using GAE. Once again, thanks!!

  • Celerontm

    >>
    jon 22 Nov, 2010
    Is English your 2nd language?
    >>

    Jon, How the fuck does it matter? And how the fuck is it relevant??? We all understood what Carlos was trying to say. I guess that’s enough.

  • chro

    I think that Java in GAE is even worse case. Consider you have only half of language, a lot of classes in JDK are blacklisted. I have no idea why Google banned some classes. So, no reports, no SOAP web services and no filesystem at all. And many frameworks fails to work depending on it. And in additional to “AS IS” statement – there is a lot of hype about GAE, and less info about limitation. Let state it clear – if you wanted to develop web Java app with defined specs and without limitations, if you previously used hot deployment, if you are not a Google fan, GAE is not for you.

  • http://n/a Chris Q

    Thanks – an interesting article.

    You are willing to admit to having made mistakes in your choice but the points you make are valid and worthwhile – certainly worth making people aware who may otherwise have rushed into the GAE because of what they “think” it is going to offer.

    Some of the “angry” comments – seemingly indignant that you are knocking GAE through your own negligence – should remember Google is big enough and has enough exposure to argue it’s case and defend it’s platform. In retrospect, a lot of the early positioning and blurb about GAE – I think – was disingenuous and misleading. I accept that is more the fault of ill informed commentators than Google.

    I hope the setbacks have not done too much harm to your business. Good luck.

  • http://top.game-rate.com GameRate

    I use GAE for more that year for some sites and social games – which have about 200K registered users.
    Of course you need to know how properly use it – but anyway GAE is good choice in many cases.

  • Noel

    For the most part I am a Microsoft stack Developer working in the enterprise, in an non MS enterprise(Java Web Sphere etc). Thanks for the insight, I have just finished reviewing various cloud offerings and thankfully came to the same conclusions you have – without spending our money :-)GAE is not really for “real” applications. But I do understand how you got sucked in to believing it is. There is very little guidence available to help understand how limited the platform is. I have several of my collegues singing the Google fanboi song and this post will go along way to backing up my conclusions. Buyer beware.

  • Pingback: Diskussion, warum die Google App Engine for Java nix ist()

  • Phil.H.

    @NOYB Tasks don’t do anything for long running processes. They are for asynchronous execution. They are subject to the same time limit of 30 seconds. Specifically for Python an exception is raised when the 30 second limit nears. They are nothing more than http calls just like those that come from a browser and have all of the same limits and restrictions. Maybe YOU should read the GAE documentation.

    http://code.google.com/appengine/docs/python/taskqueue/overview.html

    See the Task Execution section.

    Again….I LIKE GAE. So don’t ASSume I have not used it. Its simply not going to be for all projects. Theres nothing wrong with that. You might have to do some engineering to fit your idea onto the platform but thats a fun part of being a developer…at least for me….when I’m not against a deadline. 🙂

  • http://blog.refactoringin.net Danijel Arsenovski

    Carlos,
    Great job. I wonder where all this backlash comes from?

  • Adelar da Silva Queiróz

    Thanks Carlos. Very good information.

  • http://www.agost.ru Ara Minosian

    I use GAE java in my small experimental project.
    I had some problems with JSF mojarra and mail api. But now all is ok. GAE cold start became faster last month.
    Components that solved all my problems:
    1. Gae-maven-plugin
    2. JSF myfaces
    3. Objectify
    4. Removing all JDO and JPA stuff from pom file
    5. PrimeFaces

    I upload my data by admin panel like simple text and after recompile to darastore by task queue.

    1000 limit does not exist now. In next version of GAE (1.4.0) time limit for cron jobs will be removed.