Introduction to software that can be used to capture and analyse Twitter data

1. Introduction to software
that can capture data from
Twitter
Wasim Ahmed,
Information School
Email: wahmed1@Sheffield.ac.uk

2. Aims
• Disclaimer(s) to using Twitter data
• Overview of current Twitter data retrieval
and analysis software which require no
programming knowledge.
• Overview of public engagement work I
have been doing
27/08/2015 © The University of Sheffield

3. 27/08/2015 © The University of Sheffield
Tools Covered in this
Presentation
• TAGS
• NodeXL
• Mozdeh
• COSMOS Project
• Chorus

4. Ethical, privacy and copyright
issues when using Twitter data
27/08/2015 © The University of Sheffield
Read best practice guidelines

5. Refer to resources such as:
• Research using Social Media; Users’
Views link here
• COSMOS Online Guide to Social Media
Research and Ethics link here
• Unlocking the value of social media – a
review of research ethics link here
• Association of Internet Researchers
(AoIR) link here
27/08/2015 © The University of Sheffield

6. Legal issues
• Sharing of Twitter datasets is prohibited
see https://dev.twitter.com/terms/api-terms
• However, sharing Tweet IDs (to look up
the tweets used is permissible). This is
useful for reproducibility.
27/08/2015 © The University of Sheffield

7. Programming knowledge!
27/08/2015 © The University of Sheffield

8. Learn a programming language
Check these resources out to learn how to
code:
• Websites such as Code Academy
• Visit library for programming books
• YouTube Videos
27/08/2015 © The University of Sheffield

9. Why Twitter (data)?
• See my LSE impact blog post
• Twitter is a popular platform in terms of the media attention it receives and it therefore attracts
more research due to its cultural status
• Twitter makes it easier to find and follow conversations (i.e., by both its search feature and by
tweets appearing in Google search results)
• Twitter has hashtag norms which make it easier gathering, sorting, and expanding searches
when collecting data
• Twitter data is easy to retrieve as major incidents, news stories and events on Twitter tend to be
centred around a hashtag
• The Twitter API is more open and accessible compared to other social media platforms, which
makes Twitter more favourable to developers creating tools to access data. This consequently
increases the availability of tools to researchers.
• Many researchers themselves are using Twitter and because of their favourable personal
experiences, they feel more comfortable with researching a familiar platform.
27/08/2015 © The University of Sheffield

10. Different types of Twitter API
• Twitter’s Search API – focused on relevance
and not completeness, some tweets and users
may be missing from results
• Twitter Streaming API – The Streaming APIs
give developers low latency access to Twitter’s
global stream of tweet data.
• Firehose API – in theory, 100% of Twitter data
27/08/2015 © The University of Sheffield

11. How do you retrieve data?
• Use a keyword e.g., Ebola
• Use a hashtag e.g., #EbolaOutbreak
• Combine search queries using AND or OR
operators.
27/08/2015 © The University of Sheffield

12. 27/08/2015 © The University of Sheffield
TAGS – Twitter Archiving
Google Sheets
• Created and maintained by Martin Hawksey
(@mhawksey)
• TAGS is a free Google Sheet template which lets you
setup and run automated collection of search results
from Twitter.
• Set up TAGS here https://tags.hawksey.info/get-
tags/

13. 27/08/2015 © The University of Sheffield
TAGS – Twitter Archiving
Google Sheet

14. TAGS – Twitter Archiving
Google Sheet
• TAGS also allows you to visualize the
connections between users
• There is an excellent video here
27/08/2015 © The University of Sheffield

15. 27/08/2015 © The University of Sheffield
NodeXL
• NodeXL is a Microsoft Excel Plugin.
• The software can be used to obtain data from Twitter,
YouTube, and Flicker.
• NodeXL runs on Windows operating systems.
• Users can download graph options from the NodeXL graph
gallery.
• NodeXL is very easy to use – The MS Paint for network
graphs (Marc Smith)

16. 27/08/2015 © The University of Sheffield
NodeXL: example network graphs
NodeXL, example network graph of @was3210 NodeXL: Example network graph of @was3210 (using a different layout
to the graph on the left)

17. 27/08/2015 © The University of Sheffield
NodeXL tutorials
• Users can download graph options from the NodeXL Graph
Gallery (http://nodexlgraphgallery.org/Pages/Default.aspx)
• The workbooks used to create a graph (i.e., with the settings
intact) are often linked on the bottom of the page. These can
be downloaded, and further customized.
• There are some excellent NodeXL tutorials on YouTube
(https://www.youtube.com/results?search_query=NodeXl)

18. 27/08/2015 © The University of Sheffield
Mozdeh
• Mozdeh is a product of the ‘Statistical Cybermetrics
Research Group’ at the University of
Wolverhampton.
• Mozdeh is a Windows desktop program that can
gather tweets by automatically searching for
keywords associated with a topic.
• It is also very easy to use.

19. Mozdeh
27/08/2015 © The University of Sheffield
• An example time series graph of 5,055,299 tweets
related to norovirus

20. Mozdeh Tutorials
• Great user guide here
• Great theoretical overview here
27/08/2015 © The University of Sheffield

21. 27/08/2015 © The University of Sheffield
COSMOS Project
• The Collaborative Online Social Media Observatory
(COSMOS): Social Media and Data Mining is an
ESRC project a part of the strategic Big Data
investment.
• The COSMOS Project (Burnap et al, 2014) uses the
Streaming API

22. 27/08/2015 © The University of Sheffield
COSMOS Project
• Some of the features include generating:
• Word Clouds
• Frequency charts
• Network graphs
• Maps of tweets

23. 27/08/2015 © The University of Sheffield
COSMOS Project Layout

24. 27/08/2015 © The University of Sheffield
COSMOS Tutorials
• Great video tutorial(s) here

25. 27/08/2015 © The University of Sheffield
Chorus Analytics Tweetcatcher
Desktop Edition
• Chorus-TCD is a product of Brunel University.
• Uses Twitter’s Search API
• Searches as many statuses that are available from
the query at the current point of time.
• It is also very easy to use. There is a great video
introduction here.

26. 27/08/2015 © The University of Sheffield
Chorus
• This is the layout of Chorus Tweet Catcher

27. Chorus
• This is the layout of Chorus Tweet Vis
27/08/2015 © The University of Sheffield

28. Chorus Tutorials
• Chorus manual here
• Great video overview of Chorus here
27/08/2015 © The University of Sheffield

29. What if I want data going back
more than 7 days?
• In most instance you will have to pay for it
• I use Texifter(@texifter) with DiscoverText
(@discovertext)
• Can range from not that expensive to
very expensive depending on query and
time
27/08/2015 © The University of Sheffield

30. DiscoverText Tutorials
• DiscoverText explained
• You can find DiscoverText’s social data
brochure here
27/08/2015 © The University of Sheffield

31. Public Engagement
• Started to use Twitter when started my
PhD – connected with #NSMNSS and
#PhDChat community
• Started a research blog
27/08/2015 © The University of Sheffield

32. Public Engagement
Benefits of Twitter include:
• Getting tricky PhD questions answered
• Finding out about conferences
• Networking with other academics, making
new friends
7/08/2015 © The University of Sheffield

33. Public Engagement
Benefits of a blog include:
• Early feedback on PhD work – my first two
slides!
• More visibility and interest in work
7/08/2015 © The University of Sheffield

34. Map of my Twitter network
27/08/2015 © The University of Sheffield

35. Questions?
• Tweet me! @was3210
• Questions related to the tools?
• TAGS = @mhawksey
• NodeXL = @marc_smith
• COSMOS = @pbFeed
• Mozdeh = @mikethelwall
27/08/2015 © The University of Sheffield

36. To
Discover
And
Understand.