A criticism on the lack of criticism
It strikes me that one of the biggest problems hindering the growth of transmedia (and all the various things that fall under it, such as ARGs) is the absolute lack of critical looks at projects. That’s not to say that criticism doesn’t exist – it does, but it’s scattered in conversations and hidden in forum posts or mailing lists. And it is, usually, not about a project as a whole and, instead, focuses on a single issue or is a broad look at the field.
There are a number of challenges to writing critiques on projects, not the least of which is their complexity and length. It’s difficult to be critical once you’ve invested so much time and energy into a project – whether you’ve designed it or experienced it. Being critical seems harsh and, well, it might make you wonder if you’ve wasted a bunch of your time and who wants to thinks that? This is one reason why we may never have a Roger Ebert or Ben Croshaw – the commitment required to fully experience a transmedia project, especially one as complex as an ARG, is far greater than the commitment required for films and video games (or books or music or or or). To make transmedia critique a commitment on that level is difficult and, well, would require far more time than would be profitable. Which makes it a pursuit of passion or, perhaps, an academic exercise. Yet both of these color the criticism, that’s not necessarily bad, but in collaborative transmedia that ignores the “other side of the curtain.”
Which brings me to postmortems. We don’t have any publicly available – well, a few. But they are the exception and not the rule. And most aren’t extensive enough to be of much use. And, no, that pretty case study you made to show off at conferences or submit for awards does not count. In fact, I’d argue they’re part of the problem. I understand their necessity and utility, but in making and presenting them you’re looking at everything through rose colored glasses. Essentially, you’re blowing smoke up your own a… I challenge every one of you to put half the effort into creating a postmortem that you do into the case study for your next project. My bet is that you’ll see a far greater return on your investment.
Another consideration is that we’re still a small enough of community that there’s an awkwardness about saying anything that might be construed as negative… whether it’s that you don’t want to hurt your friends feelings or you hope to work with them at some point. Both of these things have stopped me on a number of occasions, I’ll admit it. I wrote an outstanding critique, I think, of the cake event for Why So Serious but I didn’t publish because I didn’t want to hurt a friend’s feelings and what if I went on to work for 42 Entertainment. Which I wound up doing and happened to help out on that same campaign… if that critique had been published? Awkward!
This reluctance isn’t doing anyone any favors. I have seen the same mistakes over and over again and I have to wonder if part of that reason is our hesitation for a bit of brutal honesty and constructive criticism. I’m not saying that we should rush out to rip projects apart, but we really should do something. We need to start having honest discussions about projects, about design decisions & implications, about the state of the industry. And we can’t be afraid to have these conversations publicly. I understand NDAs and frieNDAs – they’re important, and they shouldn’t be broken. But there is still plenty to be talked about and plenty that we can use to learn from and inform each other.
Notes:
- Links mostly go to various ARG resources/discussions which may or may not be applicable to various other transmedia, but it’s late and those are easier for me to find.
- Much credit to Christy Dena & Markus Montola who were talking about this earlier and got me thinking about it. Many of the thoughts posted here were inspired by that discussion if not taken out right from it.
Hi! I'm an experience designer specializing in transmedia storytelling & alternate reality gaming. If you want to know more about that, check out my 
email:
gtalk: 
AIM: 
LinkedIn
Facebook
Twitter
One of the main problems is that a crtique, or review, typically has a purpose in that it’s there to help an potential audience member to decide whether to take part, at least it’s that way for films and theater. For ARGs, it’s tough to do, because they’re one-offs. But yeah, I see your point and entirely agree, and have for quite a while. To read case studies, you’d think that every project was the world’s FIRST something. :)
And I for one would love to read your cake critique. :)
Good point on the purpose of a review. I know that ARGNet, for example, has struggled with that for years. How can you effectively review an ARG while it is in progress? One way, perhaps, is to critique the major events as they occur which would be a valuable resource both for creating a more comprehensive view once the experience has ended and as a review for folks curious about getting involved. Someone should get on that!
That’s less of an issue for transmedia projects that do not occur live – thinking of projects like The Matrix, Eagle Eye Freefall, and Personal Effects: Dark Arts.
And yes, most case studies are bit like youtube comments…
FIRST!!111!1!!!! BEST.ARG.EVAR OMG!!!ELEVNTYONE!
Interesting, and it does feel that post-game case studies seem to either be positive overall or are self-promoting in nature when done by the creator(s). I’d hope though that case studies also review failures and mistakes that occurred during a project, though.
But I do agree, case studies seem to have the feel of “Now that the project is done, we can disclose how happy we are about it. Even though we made X mistakes – we managed to accomplish all this!”
And like “Esteban” said, case studies seem to try to describe accomplishing some kind of the world’s “first” something; what it did to ‘innovative’ during its lifespan.
Personally, I’d love to see more discoveries, more lessons learned through problems and mistakes, citing specific examples. But the feel of the criticism really should be constructive (as hard as that may be for some projects that may generally be considered absolute failures). But coming from third parties, they can come off with a competing tone of “I would have done it this way”; as you said, more negatively, ending up hurting feelings – when that’s not the intent. It’s easy to be critical in hindsight, and it’s hard to be constructively critical, especially before the creator(s) have had a chance to come forward recognizing such problems themselves in hindsight.
Also, who is the study geared for? I tend to consider case studies as directed towards ‘creatives’ who are interested in the process and technical aspects of the project, rather than something more like an overview or review, which would be more geared towards the public, potential players and audience, etc.
Maybe the question is – what’s the etiquette for being constructively critical about someone else’s project, complete or not, in order to not come off with a sense of offense or destructive criticism?
Or should all critical case studies just end with a smiley and a bunch of XOXO’s? :P
Thoughtful post, Brooke. As an “aca-fan” of transmedia, I approach the act of “criticism” from my own specific context, so I appreciate that you’re blowing up what I normally assume when I read “criticism”! ;-)
As “Esteban” addresses in his comment, I think criticism can have a variety of uses, such as the professional industry critic, like Ebert for cinema, helping consumers to decided whether or not they might enjoy the product. As a monomedia creator (I predominantly write fiction with words on paper) who has participated in far too many undergraduate and graduate creative writing workshops (and who will be teaching them, as well), I’ve found peer criticism to be potentially quite helpful in many instances. I think, however, that peer criticism–which it seems close to what you’re advocating here for transmedia–seems to be associated with apprenticeship or the amateur creative. So, that might be a barrier and a reason for some of the closed doors and reluctance to be public with self-criticism.
To focus on the “real-time” element of many transmedia experiences, as Inquisitor Esteban mentions above, that might be one reason that transmedia/ARGs don’t receive the same type of popular critical coverage, if any at all. As I think about criticism in the perhaps academic sense, one of the issues that was recently brought up at THATCamp “Prime” with respect to writing about ARGs was the difficulty in defining the text, archiving elements. Though I wasn’t at this “unconference,” I’d imagine that one of the issues that was approached was whether to treat ARGs as a text or a performance, or both. Studying “The Beast” or “I Love Bees” might be easier, as there are extensive archives for these ARGs, but mining message boards doesn’t always yield the same level of archive, plus it relies on paratexts to resolve the text.
Thanks for the great post and the stimulating topic, Brooke!
Very interesting discussion – am reading this alongside Steve Peters blog (http://nomimes.com/newsblog/creating-buzz-vs-the-player-experience/), and Michael Anderson (http://www.argn.com/2010/06/a_call_to_action_for_alternate_reality_game_developers_play_args/)…
I think there are a couple different ways to bring about better criticism/analysis:
1. [As Jay pointed out before me] In certain writing circles there is great peer analysis – however this mostly occurs during development stage. For example, I’m primarily a writer, and if I show a screenplay or manuscript around, every man and his dog will give me an opinion without too much concern for my ‘feelings’ – however, once the work has been produced or published, it becomes a bit “precious” and people are more likely to reserve their opinion.
Learning from this: perhaps the place for peer review of an ARG is at development stage. Just as we have assessment and editing services (paid or rope-in-a-colleague), perhaps a consulting circle could be created for PuppetMasters to look over and comment on projects in development, giving the chance to point out flaws in design or story early on.
2. From a post-analysis perspective, I think good critical case studies only occur when the person doing them has something tangible to gain from the experience. Puppetmasters themselves are usually busy working towards a next project, trying to earn a living… while an academic, a journalist, an author, or a documentary filmmaker would have the objectivity and may stand to gain from dissecting the ARG if it fits within their area of interest.
The ARG industry is still relatively young, and perhaps we can learn from industries such as film where there are tonnes of magazines, websites, dvds and books on ‘how to make a film’ – by creating an open dialogue with one of these 3rd party creators from early on to analyse the ARG in one of these formats.