Wednesday, 23 September 2015

Social media research: Issues of performativity and validity

Keeva Rooney is a researcher at NatCen focusing on health surveys using bio-medical data. She has a BA (hons) in Sociology with a specialism in Social Policy from University of Warwick. During her studies, Keeva often used social media in her research, with one of her project’s focusing on online facebook groups for disabled people affected by the bedroom tax.

Social media research attracts many researchers due to its offer of practicality, creativity and accuracy which comes from its often covert nature. The idea that you can produce valid research by analysing people’s online activity is attractive to researchers looking for more innovative and contemporary ways to achieve a true insight into society. However, is the assumption that people act online as they would offline a valid one? And if not, does this raise concerns over the reliability, representativeness and overall accuracy of using social media for research purposes?

Feminist theorist Judith Butler studied how people ‘perform’ gender, be it within or outside their assigned gender norms. She believed that it was this expression of gender, not the biological sex itself, which determined gender [a]. Whilst Butler used the concept of performativity to deconstruct issues around gender, the same concept could be applied to how we use social media: do we use social media to construct a distinct identity, or is social media purely an extension of how we already perform within wider society, and does this matter for Social Media researchers?

I will be analysing this argument using two research ideologies; positivism and interpretivism [b].

On the one hand, a positivist approach suggests that social research can uncover an empirical truth about people and society. Positivists aim to find this truth in all aspects of society, using mainly quantitative data such as surveys or content analysis. For example, when analysing social media, positivists may often use sentiment analysis to analyse online attitudes and opinions of the user. For this data to be valid for understanding ‘real world’ attitudes, the opinions expressed online by social media users must be an accurate reflection of their offline beliefs. Therefore it could be argued that if someone is not being their ‘true self’ online, any conclusions drawn from this research may also not be valid.

However, it could be argued that the online persona cannot be disconnected from the offline reality. That is to say that how people act online is often a valid reflection of how they act offline. For example, a recent psychological study into internet trolling showed that those who enjoyed trolling often displayed characteristics such as sadism, psychopathy and Machiavellianism offline [c].

On the other hand, an interpretivist approach suggests that in order to analyse actions and attitudes accurately, the researcher must take the user’s interpretation of themselves as truth; when it comes to social media, how users portray themselves online is what researchers use as fact. From this perspective, any online ‘performance’ of the participant doesn’t make the research any less valid because the participant’s interpretation of their reality is always accurate.

However, what happens if people reject their online persona once it has been researched, as is often the case with people who display offensive or criminal online behaviour? As Stephen Webster’s study showed, some people will try to disassociate their online behaviour from their ‘real’ offline life, claiming that how they act online is not how they ‘really act’ [d] and some may even deny that it was them, instead saying that their account was hacked [e]. Whilst this raises several questions (such as how can you know who you are researching online), it also raises ethical concerns if someone believes that a researcher has misrepresented them by basing research solely on their online behaviour.

As the ‘troll’ studies show, online personas can often be a true reflection of offline actions and opinions. However, the extent of this may be unclear, and participants may disassociate themselves from those personas. Researchers should always consider whether social media can be used to accurately portray and analyse personal and public opinion, and if more traditional research methods should be paired with this to create a more triangulated and accurate set of data.


[a] Felluga, D: ‘Modules on Butler: On Performativity’ Introductory Guide to Critical Theory 
[b] While I acknowledge that not all researchers will fit into either ideology, I believe that analysing validity in online research can be done by discussing these two fundamental research perspectives
[c] Buckels, E.E; Trapnell, P.D; Paulhusc, D.L: ‘Trolls just want to have fun’ 
[d] Webster, S: ‘What is trolling, and why do we behave so differently online?’  
[e] Wainwright, M: ‘Man who racially abused Stan Collymore on Twitter spared prison’ 

Wednesday, 22 July 2015

Research at a click away: The inter-relational ethics of a connective methodology

Josh Jarrett is a PhD researcher as the Digital Cultures Research Centre, UWE Bristol. Josh’s research utilises an online ethnographic methodology to look the role of online play in the co-creative practices online games. Josh tweets at @Joshua_Jarrett and blogs about his research at, where this blog was originally posted.

 On Wednesday 20th May I had the pleasure to give a short overview of my research, its online methodology and some of the ethical grey areas at a University of the West of England event called ‘Ethics, Digital Data and Research using Social Media’. In this post I want to recall some of the points that were touched upon during the day, give an insight into my own online research and delve into exactly what some of the ethical grey areas are for online researchers.

MOBAs? A brief introduction to playful co-creativity

 Before I talk about specifics I should say a little more about myself and my research. I am a PhD researcher in my third year of research into the themes of online play and collaboration with the Digital Cultures Research Centre at the University of the West of England. More specifically, my research is interested in the ways online players creatively play in games and the way these acts of playful creativity carry ramifications far beyond the immediate actions of the player. For example if a player innovates a new play style in a game that had previously not been tried and this new style proves to be especially effective, how does this act of creativity carry further consequences to other stakeholders of the play space? Although it may seem like nothing particularly new for online games or acts of play more widely it is within these dynamics that one of the most played online games genres, the Multiplayer Online Battle Arena (MOBA), is underpinned. Known for games such as League of Legends or Dota 2, the MOBA genre has come to define the games and digital landscape in significant ways through popularising live streaming on sites such as, giving rise to a thriving worldwide electronic-sports industry and introducing new models of fair ‘free to play’ allowing vast numbers of players to play. All of these trends have meant the MOBA space is one laden in different stakeholders, from its millions of daily players, its worldwide network of professional players who make a living off playing and its developers who ultimately seek to monetise all of this activity. My research looks at the role of online play in co-creating this genre and more critically, how the differing power dynamics between stakeholders affects their respective values in the play space. For this post I want to leave all of that to the side however and talk a bit about how I go about ethnographically grasping this vast and dynamic online culture.

A connective Reddit methodology  

 My methodology consists of three distinct strands of data collection. These are auto-ethnographic in-game experiences, (professional) player interviews and open Reddit discussions. All of these research methods take place online and what is interesting is how they all share a connected relationship. Although my main source of data collection is open discussions on the news and social networking site Reddit (which is the focus of this post), it is vital to consider the methodology as a whole.

 The auto-ethnographic element to this research is similar to many studies of games, an essential element that informs nearly all of the work. As a research method auto-ethnography has been employed in numerous online games (Boellstorff et al, 2012) and it informs the questions posed in wider dialogues with players in Reddit spaces. However the auto-ethnography also serves as a window into more than just knowledge of the game and its culture but it also serves as a mode of authenticity when posing questions to players.

 A recurring theme throughout the day was how different online platforms share a connected relationship and can often bypass the privacy settings of one through the other, for example in case of Storify and Twitter that Kandy Woodfield touched upon. I find this example pertinent to the context of my own research as it taps into what Dijck (2013) has termed the ‘connective’ context of social media, which is to say the interrelated and ecological relationship different platforms share with each other online. Storify for example, is a site that heavily relies upon Twitter feeds to construct its content and in doing so the ethics of how to use Twitter must also consider Storify’s interrelation. In my own research this ‘connective’ context is one that is heavily woven into everything I do. By a large margin the open discussion format in Reddit is my main mode of data collection however it is largely enabled through my wider auto-ethnographic experience.

 When opening up a discussion with participants upon Reddit I am wholly transparent about my status as a researcher and always link my blog and state I am open to questions myself. In addition, I also introduce myself as a player through linking my in-game profile and stating an example of what I am talking about from my own experience. For players who are in a space to talk with other players and not expecting questions posed by an academic researcher this further introduction of myself as a fellow player is significant and opens up a much more casual, intuitive, and insightful response to my questions than it would if the questions were strictly formal. The status of the researcher as a player is especially important here as Reddit’s format can be extremely resistant to researchers due to the architecture of the platform.

 Reddit is similar to a forum in many ways and it works through the same persistent threads of conversation often heavy in memes and external links. The vital difference between Reddit and a forum is its system of up-voting and down-voting posts that leads to what many have dubbed a ‘Hive Mind’ whereby only certain types of posts are up-voted and therefore visible for the vast majority of users. In practice this means getting attention to a research question posed on Reddit can be difficult if, for example, people perceive your reasons for being in the space as not similar to theirs. In my experience of making threads in the /r/leagueoflegends sub-Reddit that has thousands of active users at any given time, attention to these dynamics has proven essential.  If a post does not get up-votes quickly it will sink and essentially vanish from users; so introductions are important! A connective sense of authenticity (I.E. auto-ethnographically playing the game as well as externally researching it) is one way of gaining traction here and it’s this kind of wider attention to online platforms that is an essential consideration throughout my online methodology.

Connective ethics

 Similar to the Twitter / Storify example touched upon above, a space such as Reddit must be considered in relation to other platforms when considering the ethical implications of the research. If for example, you state not to use the name of participants as a measure of protection against their identity (even avoiding use of their pseudonyms) but you quote their responses from an open discussion, it is very easy for a search engine such as Google to identify the quote and take you to the page where the discussion happened. Throughout the ethics in social media research day similar examples of these interrelations between platforms undermining ethical standards were touched upon and just as the relationship between Reddit and Google is here problematic, so too are numerous other examples. Potential solutions to this particular ethical concern included closing threads after conversations end and paraphrasing quotes (as is often done when working with children) from participants to avoid search engine detection, however there is no clear or effective answer here.

As with many other researchers working with online participants, I do not have an answer to unravelling a universal code of ethical standards here. The interrelations between online platforms are constantly changing as they respond to a variety of socio-technical developments and as a researcher, the only thing I can say with any certainty is that attention to these dynamics is essential. A particularly connective methodology such as my own that I have touched upon here points towards the opportunities inherent in online research as much as it sketches out potential ethical grey areas. There might not be any clear or definitive answer when approaching these grey areas but nonetheless they need to be sketched out and carefully considered. Research at a click away is exciting, insightful and potentially rich as much as it can be problematic.


Boellstorff, T. Nardi, B. Pearce, C. and Taylor, T, L. (2012) Ethnography and Virtual Worlds: A Handbook of Method, Oxford: Princeton University Press.
Dijck, V, J. (2013) Culture of Connectivity: A Critical History of Social Media, New York: Oxford University Press.