entering 2011 IGF or not?

Discussion in 'Indie Business' started by m3xican, Oct 16, 2010.

  1. ionside

    ionside New Member

    Joined:
    Mar 27, 2008
    Messages:
    283
    Likes Received:
    0
    This is exactly what I've been weighing up. My wife is insistent on me submitting to igf this year though. I'm starting to suspect she's slept with a few of the judges.
     
  2. Yshaana

    Yshaana Guest

    wow, if they can't be arsed to play a cool looking original single player game, there is no way they would test a multiplayer only title.

    I agree with Jim Buck, I would call them out on this. Even if your game might not win, it is still unfair and biased, which is the opposite of what any award/contest should be.

    I'll wait next year anyway to see if we should enter Fray to the contest as we are too early in dev, but I would be pissed off if no one actually took the time to try out the game. A sad day because your games are pretty fresh Cas and Cliffsky, and definetly deserve recognition.
     
  3. papillon

    Indie Author

    Joined:
    Jul 26, 2004
    Messages:
    2,193
    Likes Received:
    0
    One of the reasons I wanted to judge was to find out exactly how the system worked in case I ever wanted to submit again. They certainly are SUPPOSED to play every game (or at least give it a good try and then beg for mercy if the blasted thing absolutely will not run, which happens) and the IGF runners really want that to happen, and really want the judges to not only provide feedback but to have insightful discussions about the games with each other.

    Except that the judges are volunteers scattered around the globe, and busy, and some of them barely remember the contest is happening until the last minute, and when invited to discuss things with each other fidget like kids the teacher's just called on, saying nothing unless someone else starts it.

    Any judge actually caught not playing the games at all will not be invited to judge again, but what else can they do if someone chooses to be a jerk? It's not like the IGFpeeps can actually sit over each judge's shoulder and force/watch them, instead they keep trying to refine the system to encourage people to behave. As I understand it. I am absolutely not talking for them officially here.
     
    #23 papillon, Oct 17, 2010
    Last edited: Oct 18, 2010
  4. zoombapup

    Moderator Original Member

    Joined:
    Nov 25, 2004
    Messages:
    2,890
    Likes Received:
    0
    See, thats what I dont get. I mean Andy won with a 4 player game. If you can't get people to play a game, how the hell do you manage to get 4 and if you didnt have 4, then how valid is the judging of the game?
     
  5. Jasmine

    Jasmine New Member

    Joined:
    Aug 8, 2010
    Messages:
    195
    Likes Received:
    0
    I've organised game making contests in the past, and I know that judges are hard to come by. Beggars can't be choosers, so you take the good with the bad.

    Only when you have them in surplus can you afford to pick and choose based on past experience.

    "This is a good judge, who takes the role responsibly, is patient getting the games to run, makes a good effort to play them, and makes fair and thoughtful reviews afterward. I'd like this person to be a judge again."
     
  6. papillon

    Indie Author

    Joined:
    Jul 26, 2004
    Messages:
    2,193
    Likes Received:
    0
    If it was multiplayer that did NOT require special registration codes, judges with local friends and family could arrange their own games in their homes, which would help a lot. If by luck you were assigned two judges who knew each other and could coordinate, that would also make a difference, or if you got a really outgoing judge who could stir people up. Beyond that I can only guess that perhaps some games are So Cool Looking that it gets people past the shyness barrier of not talking to each other.

    Other people were certainly playing the games I was assigned, but mostly didn't talk about them unless they were having problems. But it varies! Who you get is random, luck could be with you or against you.

    The new judging system, dropping the numerical comparisons by random judges, will hopefully turn out to be more fair. (two people will rate a 'good' game with entirely different numbers even if they liked it about the same, number ratings end up meaningless)
     
    #26 papillon, Oct 17, 2010
    Last edited: Oct 17, 2010
  7. Applewood

    Moderator Original Member Indie Author

    Joined:
    Jul 29, 2004
    Messages:
    3,859
    Likes Received:
    2
    WTF?
    I didn't realise you were one of the judges, but for you to start going on about the panel not looking at games and mentioning "evil" judges, I think makes the whole thing a travesty.

    I'd expect better treatment than you describe from a blog or review site, but the picture you're painting here of what is essentially a paid subscriber service is a joke. If I pay $100 to have my product judged and nobody even looks at it, that's not just bad service - it's fraud.

    I guess I won't be entering this thing after all.
     
  8. papillon

    Indie Author

    Joined:
    Jul 26, 2004
    Messages:
    2,193
    Likes Received:
    0
    Um, huh? I've been saying people DO look at the games. Only cas and cliff have been claiming they didn't.

    I've been pointing out that it's INDIVIDUAL VOLUNTEERS and therefore it is possible for an individual to misbehave and that this is not really the fault of the IGF as a whole, nor anything they can control. Sorry if I wasn't sufficiently clear. (But if you have proof that nobody played your game, DO complain about that so that they know those judges were problematic.)

    The thing I was complaining about with a different contest was the Baftas, not the IGF. I read the application information and discovered that you weren't even asked to SUBMIT your game unless you made it to round two, which I thought was absolutely ridiculous. Especially as you had to pay large sums of money to enter round one.

    The IGF is not evil, they're doing the best they can, but they can't force the judges to be perfect. The new system they're trying this year should hopefully encourage people to pass games to better-qualified judges for individual areas.
     
    #28 papillon, Oct 17, 2010
    Last edited: Oct 17, 2010
  9. Applewood

    Moderator Original Member Indie Author

    Joined:
    Jul 29, 2004
    Messages:
    3,859
    Likes Received:
    2
    Well, there's this.

    And a final point that charging money for something makes it a professional contract and there's a reasonable expectation that the people on the other end are behaving in a professional manner. That's not what I'm getting from this, but maybe that's just me.

    Um. Yes it is. It's certainly not mine or cliffskis.
     
    #29 Applewood, Oct 17, 2010
    Last edited: Oct 17, 2010
  10. papillon

    Indie Author

    Joined:
    Jul 26, 2004
    Messages:
    2,193
    Likes Received:
    0
    Okay, understood. Take my comments with a grain of salt, though, as I really have only done this once. :)

    The developers are paying money, but the judges are unpaid volunteers, and sometimes getting volunteers to do things on time can be a pain, ask anyone who's tried to manage a group of reviewers. I got the impression that there were some people lagging (not starting their reviews until quite late) and/or not giving back as much as the organisers would have liked them to. It is not a perfect system, and you are not guaranteed that your randomly-assigned judge will devote days to discovering every nuance of your game. There quite possibly are better uses for your $100.

    But they definitely want their judges to do a good job, which is more than I can say for some contests.


    It's certainly not yours or cliffski's, no! But I'm not sure how they can be expected to make it absolutely perfect either. They're trying, from what I've heard they're trying every year to improve the judging process. The new jury system should hopefully result in a more in-depth review for at least the games that make it past the first cut.

    If seriously nobody played a game that was submitted, that's horrible, that shouldn't happen, and I dont know why it did. Multiple people are assigned to every game; for really no one to play it would require that multiple people picked at random all turned out problematic.

    EDIT also to add link to some judges talking about the process:
    http://www.gamasutra.com/view/news/26685/Opinion_Demystifying_the_IGF_Judging_Process.php
     
    #30 papillon, Oct 17, 2010
    Last edited: Oct 18, 2010
  11. Matthew

    Indie Author

    Joined:
    Jul 29, 2004
    Messages:
    493
    Likes Received:
    0
    Every game last year was played by more than one judge. It is not possible for games to fall through the cracks; we have extensive oversight into the statistical part of the process. The average game had something like 6 scores (I don't have the database at my fingertips right now or I'd pull some stats). The average judge--and there were over 150--had 15 games assigned to them. This number fluctuated a little depending on the judges' available console/etc hardware.

    Last year there were 600 games across main, student, and mobile, and 200+ judges across all three categories. Yes, it is logistically difficult to run the IGF, and yes, things are complex enough for chance to play a role.

    My personal opinion:

    * If you're looking at IGF as a marketing avenue, and you have an existing fan base and are a well-known developer, then yeah, I'm not sure if you're going to get $100 of sales out of it. I don't think $100 in, $100+ out is a useful way to view the IGF, if all you are counting is dollar amount.

    * If you're trying something new, and looking to find your market, or looking for press coverage to help you find that market, then $100 probably makes sense. If you're looking for motivation to sprint towards a deadline, and are on the cusp of making something awesome, then it's definitely a good deadline.

    * If you're looking for exposure to parlay into console negotiations, potential publisher/funder negotiations, etc, then the possible prestige of being a finalist is certainly worth it.

    * The feedback issue is interesting to me. Last year some developers received several pages of feedback, some only short, single comments. Feedback isn't the stated goal of the IGF. And I think there are misconceptions here--judges do not have any special insight into games. It's not like judges are diamond appraisers, who can tell you things about a diamond you don't have the training or equipment to discern for yourself. Judges are people who play your game and base their opinion on what it was like to play your game. That's all :p
     
  12. simonc

    Original Member

    Joined:
    Jul 5, 2005
    Messages:
    17
    Likes Received:
    0
    Just got alerted to this thread - Brandon Boyer, who is IGF Chairman this year, is stuck in the '48 hours to approve new members' so can't comment. I'm Simon Carless, and I've run IGF in various forms over the past few years, I'm Chairman Emeritus nowadays.

    So, a few notes:

    - As to folks like Cliff and Cas who say that their games were insufficiently judged: all I can say is that we had over 170 judges last year, and there were always at least 6 or 7 judges assigned per game - and to my knowledge, there were almost no games in the Main Competition judged by less than 5 players.

    Last year, with compulsory comments, we also had almost 2000 comments in total for the 300 Main Competition games - a LOT more than previous years. Obviously, I'm not sure which years you guys entered, and I'm also not sure how people played your games - some people might have had full versions already, since a lot of the judges are members of the indie community and might be au fait with your titles.

    All I can say is that there were an average of 6 detailed comments per game in 2010, and you can't do that without having played a game in-depth. This year we're letting judges 'recommend' to particular discipline-specific juries.

    So I would definitely recommend entering if you have, for example, kickass AI - there will be a jury of coders who will be looking closely at recommended games, and this should get much more focused results than previous years for discipline-specific prizes.

    - I actually don't have a problem with some of the comments in here, flip though they may be. If you're looking for sales and money is limited, you absolutely might be better spending the money on advertising. Although people do browse the IGF games directory, there is relatively little precedent to people selling exponentially more games just through entering the IGF. (However, by entering people like IndieGames blog and Kotaku may notice it and do larger pieces on it. Or you could just contact them yourselves...)

    So I do believe that we've been behaving in a professional manner, and I do believe that if you enter your game, it will get assigned to a decent number of judges, and the majority of those judges will play it. We do look at who has been playing games from year to year and cycle our judges as a result.

    Has the IGF always been perfect in the past? Nope. But I do feel like that, if you enter your game, a goodly amount of people will play it, they will try to give you written feedback that may help you, and you certainly have a chance of being a finalist, which could be a positive thing for your game from a publicity point of view.

    We care about helping indies, which is why we're doing the festival and awards. It certainly doesn't make us tonnes of money and it certainly can engender hostility - as well as positivity - from the community. But we're committed to getting it right, and I do believe that high-quality 'regular' games can do well as well as the more art-styled games, which we are trying to move more into the Nuovo section of the awards.

    [EDIT: Oop, Matthew turned up at the same time and said some similar things, but hey, reinforcement!]
     
  13. m3xican

    m3xican New Member

    Joined:
    May 5, 2009
    Messages:
    95
    Likes Received:
    2
    Wow! Two replies from Matthew Wegner and Simon Carless, that's pretty cool :)

    That's exactly what we're aiming for. We're in! ;)
     
  14. princec

    Indie Author

    Joined:
    Jul 27, 2004
    Messages:
    4,873
    Likes Received:
    0
    What we weren't seeing is a connect between that $100 and a legion of unpaid voluntary judges.

    For $100, we want something more than an email that says "No." in return. I can get that by emailing Valve whenever I like. For $100, we want coverage. Even if it's coverage that says "this game didn't make the grade, but here's a link to it anyway." in a list of 500 other games that didn't make the grade.

    Cas :)
     
  15. papillon

    Indie Author

    Joined:
    Jul 26, 2004
    Messages:
    2,193
    Likes Received:
    0
    And again, I apologise if my hypothesizing about how some game in the past might have gotten less judge attention gave anyone the wrong idea. :) I definitely still think the IGF is a positive thing!

    Well, there is the list of all games entered? People do browse it...
     
  16. Moose2000

    Moose2000 New Member

    Joined:
    Jul 3, 2006
    Messages:
    138
    Likes Received:
    0
    On the strictly financial reckoning, if you have an iPhone / iPad title and get through to winner / nominee / honourable mention, then you get featured on the App Store. (Well, the past 2 years they've done that, anyway). You get very clear stats on what that does for your sales - it certainly paid for itself many times over for me.

    It is also a prestigious award, and even as a not-quite-finalist I've found that very useful for getting the attention of useful people.
     
  17. simonc

    Original Member

    Joined:
    Jul 5, 2005
    Messages:
    17
    Likes Received:
    0
    In previous years (before 2010), our comments have not been that extensive, I agree. We had a LOT of comments in 2010, and many of them were very detailed - people posted theirs publicly on the TIGSource forums if people want to go search them out.

    Can't guarantee they will be THAT extensive every year, but some people got 1000+ words in 2010. Not sure when you last entered, Cas?

    As for 'a list of X games', we are going to be heavily promoting the list of Main Competition entrants - probably more so than previous years, including banner ads and newsletter inclusions across our network.

    Also, to be clear, what the list of entrants does is allow a lot of people - some of them journalists - to look through and see what catches their eye. But obviously, certain types of games will stick out more, and if you think your game is the kind that won't do that - again, that's a point against signing up.

    (Nonetheless, we've always been the only indie contest that sets it up so you can upload a screenshot and video and people can check out all the entrants, no matter how well they do - and we're continuing that.)

    Going back to the money, if you think you can spend the $95 a better way, that's fine - we're not making tonnes of money off this. In fact I don't think the combined entrants' fees will even quite pay for the IGF Pavilion at GDC, let alone the IGF Awards - though we have sponsors that squeak us toward breakeven. So.. that's how we're trying to do things.
     
    #37 simonc, Oct 19, 2010
    Last edited: Oct 19, 2010
  18. vjvj

    Indie Author

    Joined:
    Sep 25, 2004
    Messages:
    1,732
    Likes Received:
    0
    This is a good point, and I have a feeling that most "3rd-party acquisitions"-types also make it a habit to go through the whole list regardless of who gets nominated.
     
  19. meds

    meds New Member

    Joined:
    Dec 26, 2008
    Messages:
    344
    Likes Received:
    0
    I don't like that I have to pay money so that a bunch of random nubs look at it and maybe vote for it if the wind is blowing in the right direction.

    The only reason the IGF is regarded as prestigious is because it's the only competition of its kind around, in any other industry the IGF would be laughed at. It's sad that so many indie game developers look to it as a source of prestige, says a lot about the industry.
     
  20. papillon

    Indie Author

    Joined:
    Jul 26, 2004
    Messages:
    2,193
    Likes Received:
    0
    I think you may be highly over-idealising other industries. :)
     

Share This Page

  • About Indie Gamer

    When the original Dexterity Forums closed in 2004, Indie Gamer was born and a diverse community has grown out of a passion for creating great games. Here you will find over 10 years of in-depth discussion on game design, the business of game development, and marketing/sales. Indie Gamer also provides a friendly place to meet up with other Developers, Artists, Composers and Writers.
  • Buy us a beer!

    Indie Gamer is delicately held together by a single poor bastard who thankfully gets help from various community volunteers. If you frequent this site or have found value in something you've learned here, help keep the site running by donating a few dollars (for beer of course)!

    Sure, I'll Buy You a Beer