Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Iswc2011 role-composition-analysis

10th International Semantic Web Conference (ISWC 2011), Bonn, Germany
(http://iswc2011.semanticweb.org/)
In this event, the OU team presented their work towards modelling and
analysing user behaviour in online communities. The goal of this work
is to monitor and capture member activities and to analyse emerging
behaviour over time. This provides the policy maker the ability to
focus on smaller and more manageable groups of users.

  • Login to see the comments

  • Be the first to like this

Iswc2011 role-composition-analysis

  1. 1. Modelling and Analysis of User Behaviour in Online Communities Sofia Angeletou, Matthew Rowe and Harith Alani Knowledge Media Institute, The Open University, Milton Keynes, United Kingdom International Semantic Web Conference 2011. Bonn, Germany. 2011
  2. 2. The Utility of Online Communities• Online communities yield value in terms of: – Idea generation – Customer support – Problem solving• Managing and hosting communities can be – Expensive – Time-consuming• Large investments in communities, therefore they must: – flourish and remain active – remain… ‘healthy’Modelling and Analysis of User Behaviour in Online 1Communities
  3. 3. Increasingly Active Community What did the community look like at the point?Modelling and Analysis of User Behaviour in Online 2Communities
  4. 4. Increasingly Inactive Community What were the conditions at this point?Modelling and Analysis of User Behaviour in Online 3Communities
  5. 5. Gauging Health• How can we gauge community health? – Post Count? – User Count? – Communication/Interaction? – Behaviour?• Domination of one behaviour could lead to churn – Preece, 2000• Behaviour in online community is influenced by the roles that users assume – Preece, 2001• To provide health insights we need to monitor behaviour over time – Combined with basic health metrics (e.g. post count)Modelling and Analysis of User Behaviour in Online 4Communities
  6. 6. Supporting Community Owners1. Monitor and capture member activities2. Analyse emerging behaviour over time3. Understand the correlation of behaviour with community evolution4. Learn when to intervene to influence the communityModelling and Analysis of User Behaviour in Online 5Communities
  7. 7. Supporting Community Owners1. Monitor and capture member activities2. Analyse emerging behaviour over time3. Understand the correlation of behaviour with community evolution4. Learn when to intervene to influence the communityModelling and Analysis of User Behaviour in Online 6Communities
  8. 8. Contributions• Ontology to model behavioural roles and behaviour features – Capturing time stamped user attributes• Method to infer user roles in online communities – Using semantic rules• Analysis of community health through role composition – Identifying composition patterns for healthy communitiesModelling and Analysis of User Behaviour in Online 7Communities
  9. 9. Outline• Behaviour Ontology• Behaviour Features• Community Roles• Approach for Behaviour Analysis – Constructing Semantic Rules – Applying Semantic Rules• Analysis of Community Health• Predicting Community Health• Findings• Future Work• ConclusionsModelling and Analysis of User Behaviour in Online 8Communities
  10. 10. Behaviour Ontology http://purl.org/net/oubo/0.3Modelling and Analysis of User Behaviour in Online 9Communities
  11. 11. Behaviour Features• In-degree Ratio – Proportion of users that reply to user ui• Posts Replied Ratio – Proportion of posts by ui that yield a reply• Thread Initiation Ratio – Proportion of threads started by ui• Bi-directional Threads Ratio – Proportion of threads where ui is involved in a reciprocal action• Bi-directional Neighbours Ratio – Proportion of ui‘s neighbours with whom a reciprocal action has taken place• Average Posts per Thread – Mean number of posts in the threads that ui has participated in• Standard Deviation of Posts per Thread – Standard deviation of posts in the threads that ui has posted inModelling and Analysis of User Behaviour in Online 10Communities
  12. 12. Community Roles Elitist Grunt Joining Conversationalist Popular Initiator Popular Participant Supporter Taciturn Ignored Jeffrey Chan, Conor Hayes, and Elizabeth Daly. Decomposing discussion forums using common user roles. In Proc. Web Science Conf. (WebSci10), Raleigh, NC: US, 2010.Modelling and Analysis of User Behaviour in Online 11Communities
  13. 13. Community Roles Elitist Grunt Joining Conversationalist Popular Initiator Popular Participant Supporter Taciturn Ignored Jeffrey Chan, Conor Hayes, and Elizabeth Daly. Decomposing discussion forums using common user roles. In Proc. Web Science Conf. (WebSci10), Raleigh, NC: US, 2010.Modelling and Analysis of User Behaviour in Online 12Communities
  14. 14. Community Roles Elitist Grunt Joining Conversationalist Popular Initiator Popular Participant Supporter Taciturn Ignored Jeffrey Chan, Conor Hayes, and Elizabeth Daly. Decomposing discussion forums using common user roles. In Proc. Web Science Conf. (WebSci10), Raleigh, NC: US, 2010.Modelling and Analysis of User Behaviour in Online 13Communities
  15. 15. Community Roles Elitist Grunt Joining Conversationalist Popular Initiator Popular Participant Supporter Taciturn Ignored Jeffrey Chan, Conor Hayes, and Elizabeth Daly. Decomposing discussion forums using common user roles. In Proc. Web Science Conf. (WebSci10), Raleigh, NC: US, 2010.Modelling and Analysis of User Behaviour in Online 14Communities
  16. 16. Community Roles Elitist Grunt Joining Conversationalist Popular Initiator Popular Participant Supporter Taciturn Ignored Jeffrey Chan, Conor Hayes, and Elizabeth Daly. Decomposing discussion forums using common user roles. In Proc. Web Science Conf. (WebSci10), Raleigh, NC: US, 2010.Modelling and Analysis of User Behaviour in Online 15Communities
  17. 17. Community Roles Elitist Grunt Joining Conversationalist Popular Initiator Popular Participant Supporter Taciturn Ignored Jeffrey Chan, Conor Hayes, and Elizabeth Daly. Decomposing discussion forums using common user roles. In Proc. Web Science Conf. (WebSci10), Raleigh, NC: US, 2010.Modelling and Analysis of User Behaviour in Online 16Communities
  18. 18. Community Roles T abl e 1. Roles and t he feat ure-t o-level mappings R ol e Feat ur e L evel E l i t i st I n-D egr ee R at i o l ow B i -di r ect i onal T hr eads R at i o hi gh B i -di r ect i onal N ei ghb our s R at i o l ow G r unt B i -di r ect i onal T hr eads R at i o m ed B i -di r ect i onal N ei ghb our s R at i o m ed A ver age Post s p er T hr ead l ow ST D of Post s p er T hr ead l ow Joi ni ng Conver sat i onal i st T hr ead I ni t i at i on R at i o l ow A ver age Post s p er T hr ead hi gh ST D of Post s p er T hr ead hi gh Popul ar I ni t i at or I n-D egr ee R at i o hi gh T hr ead I ni t i at i on R at i o hi gh Popul ar Par t i ci pant s I n-D egr ee R at i o hi gh T hr ead I ni t i at i on R at i o l ow A ver age Post s p er T hr ead m ed ST D of Post s p er T hr ead m ed Supp or t er I n-D egr ee R at i o m ed B i -di r ect i onal T hr eads R at i o m ed B i -di r ect i onal N ei ghb our s R at i o m ed T aci t ur n B i -di r ect i onal T hr eads R at i o l ow B i -di r ect i onal N ei ghb our s R at i o l ow A ver age Post s p er T hr ead l ow ST D of Post s p er T hr ead l ow I gnor ed Post s R epl i ed R at i o l owModelling and Analysis of User Behaviour in Online 17Communities
  19. 19. Constructing RulesStructural, social network, Feature levels change with thereciprocity, persistence, participation dynamics of the communityRun rules over each user’s features Based on related work, we associateand derive the community role composition roles with a collection of feature-to-level mappings e.g. in-degree -> high, out-degree -> high Modelling and Analysis of User Behaviour in Online 18 Communities
  20. 20. Applying RulesCONSTRUCT { ?role a ?t . ?this social-reality:count_as ?role . ?context a social-reality:C . ?role social-reality:context ?context . ?temp a oubo:TemporalContext . ?forum a sioc:Forum . ?forum oubo:belongsToContext ?context . ?temp oubo:belongsToContext ?context} WHERE { BIND (oubo:fn_getRoleType(?this) AS ?type) . BIND(smf:buildURI("oubo:Role{?type}") AS ?t) . .....}Modelling and Analysis of User Behaviour in Online 19Communities
  21. 21. Applying RulesCONSTRUCT { ?role a ?t . ?this social-reality:count_as ?role . ?context a social-reality:C . ?role social-reality:context ?context . ?temp a oubo:TemporalContext . ?forum a sioc:Forum . ?forum oubo:belongsToContext ?context . ?temp oubo:belongsToContext ?context} WHERE { BIND (oubo:fn_getRoleType(?this) AS ?type) . BIND(smf:buildURI("oubo:Role{?type}") AS ?t) . .....} 1. SPIN function fn_getRoleType() matches the user (?this) with the relevant role typehttp://spinrdf.org/spin.htmlModelling and Analysis of User Behaviour in Online 20Communities
  22. 22. Applying RulesCONSTRUCT { ?role a ?t . ?this social-reality:count_as ?role . ?context a social-reality:C . ?role social-reality:context ?context . ?temp a oubo:TemporalContext . ?forum a sioc:Forum . ?forum oubo:belongsToContext ?context . ?temp oubo:belongsToContext ?context} WHERE { BIND (oubo:fn_getRoleType(?this) AS ?type) . BIND(smf:buildURI("oubo:Role{?type}") AS ?t) . .....} 2. Build the URI for the behaviour role class of the user, based on the ?type matchModelling and Analysis of User Behaviour in Online 21Communities
  23. 23. Applying RulesCONSTRUCT { ?role a ?t . ?this social-reality:count_as ?role . ?context a social-reality:C . ?role social-reality:context ?context . ?temp a oubo:TemporalContext . ?forum a sioc:Forum . ?forum oubo:belongsToContext ?context . ?temp oubo:belongsToContext ?context} WHERE { BIND (oubo:fn_getRoleType(?this) AS ?type) . BIND(smf:buildURI("oubo:Role{?type}") AS ?t) . .....} 3. The user (?this) is associated with the role in the given time span (?temp) and forum (?forum)Modelling and Analysis of User Behaviour in Online 22Communities
  24. 24. Analysis of Community Health• How is community role composition associated with activity?• Dataset – Irish community message board: Boards.ie – All posts used from 2004 – 2006 – Selected 3 forums for analysis • F246: Commuting and Transport • F388: Rugby • F411: Mobile Phones and PDAs• Measured at 12-week increments: – Forum composition (% of roles) • E.g. 20% elitists, 10% grunts, etc – Number of postsModelling and Analysis of User Behaviour in Online 23Communities
  25. 25. Analysis: Results (1) Forum 246 – Commuting and TransportModelling and Analysis of User Behaviour in Online 24Communities
  26. 26. Analysis: Results (2)Forum 246 – Commuting Forum 388 – Rugby Forum 411 – Mobile Phones and Transport and PDAs Modelling and Analysis of User Behaviour in Online 25 Communities
  27. 27. Analysis: Results (3) Forum 246 – Commuting and TransportModelling and Analysis of User Behaviour in Online 26Communities
  28. 28. Analysis: Results (4)Forum 246 – Commuting Forum 388 – Rugby Forum 411 – Mobile Phones and Transport and PDAs Modelling and Analysis of User Behaviour in Online 27 Communities
  29. 29. Predicting Community Health• Can we predict community health from role composition?1. Predict either an increase or decrease in activity – Features: roles and percentages – Class label: increase/decrease – Performed 10-fold cross validation with J48 decision tree2. Predict post count from role composition – Independent variables: roles and percentages – Dependent variable: post count – Induced linear regression model and assessed the modelModelling and Analysis of User Behaviour in Online 28Communities
  30. 30. having eit her increased (pos) or decreased (neg) since t he previous t ime window.For our classificat ion t ask we used t he J48 decision t ree classifier in a 10-foldcross validat ion set t ing (due t o t he Prediction: dat aset s) by: first, iden- limit ed size of t he Results (1)t ifying increases and decreases in each of t he forums, and secondly, ident ifyingact ivity changes across communit ies, by combining forum dat aset s t oget her int oa single dat aset . To report on t he performance of our approach we used preci-sion, recall, f-measure (set t ing β = 1) and t he area under t he Receiver Operat orCharact erist ic Curve (ROC). T ab l e 2. Result s from det ect ing changes in act ivity using community composit ion For um P R F1 ROC 246 0.799 0.769 0.780 0.800 388 0.603 0.615 0.605 0.775 411 0.765 0.692 0.714 0.617 A ll 0.583 0.667 0.607 0.466 Table 2 present s t he result s from our classificat ion experiment s. For forum246 we achieve t he highest F1 value due t o t he act ivity in t he forum st eadilyincreasing over t ime and t he precision value indicat ing t hat in t his forum t hecomposit ion pat t erns account for fluct uat ions in act ivity. For forum 388 we re-turn t he lowest F1 value, indicat ing t hat t he variance in act ivity renders t hepredict ion of act ivit y increase difficult wit hin t his forum, t his could possibly Modelling and Analysis of User Behaviour in Online 29be due t o t he seasonal fluct uat ions in int erest surrounding t he rugby season. Communities
  31. 31. t his analysis we have ident ified four key take-home messages: 1. Healt hy communit ies cont ain more elit ist s and popular part icipant s. 2. Unhealt hy communit ies cont ain Prediction: Results (2) many t acit urns and ignored users. 3. Communit ies exhibit idiosyncrat ic composit ions, t hus reflect ing t he differing dynamics t hat are required/ exhibit ed by individual communit ies. 4. A st able composit ion, wit h a mix of roles, increases community healt h. T ab l e 3. Linear regression model induced from t he forum composit ion of f388 R ol e E st ’ Coeffici ent St andar d E r r or t -Val ue P ( x > t ) Joi ni ng Conver sat i onal i st 69.20 43.82 1.579 0.1751 Popul ar I ni t i at or s 173.41 54.72 3.169 0.0248 * * T acit ur ns -135.97 101.91 -1.334 0.2397 Supp or t er s -266.53 109.60 -2.432 0.0592 * E l i t i st s -105.19 55.88 -1.882 0.1185 Popul ar Par t i ci pant s 372.44 103.24 3.608 0.0154 * * I gn or ed -75.69 33.39 -2.267 0.0727 * 2 Sum m ar y: R es. St E r r : 311.5, A dj R : 0.8514, F 7 , 5 : 10.82, p-val ue: 0.0092 Si gni f. codes: p-val ue < 0.001 * * * 0.01 * * 0.05 * 0.1 . 15 D iscussion and Fut ur e W or kT he communit ies we chose t o analyse in t his paper were forums from Boards.ie.It is possible of course t hat different behavioural pat t erns could emerge when Modelling and Analysis of User Behaviour in Online 30 Communitiesanalysing different communit ies. However, t here is no reason t o assume t hat our
  32. 32. Findings1. Active communities contain more Elitists and Popular Participants =2. Unhealthy community contain more Tactiturns and Ignored users =3. Communities exhibit idiosyncratic compositions4. A stable, mixed composition increases activityModelling and Analysis of User Behaviour in Online 31Communities
  33. 33. Future Work• Micro-level role analysis – Development of a ‘role lifecycle’• Identification of key community users – To avoid such users ‘churning’• Explore alternative methods for role labelling – Current approach misses ~29% of users• Extend analysis to other community types – Enterprise communities – Social networking platformsModelling and Analysis of User Behaviour in Online 32Communities
  34. 34. Conclusions• Presented an approach to label users with roles based on their behaviour – Ontology captures user behaviour as numeric attributes – Semantic rules are employed to infer user roles• Behaviour roles are only a subset of the literature – Roles differ based on the community type – Our approach is portable to other roles• Correlated community composition with activity – Increase in Elitists and Popular Participants = increased activity – Increase in Taciturns and Ignored = decreased activity – Stable, mixed composition = increased healthModelling and Analysis of User Behaviour in Online 33Communities
  35. 35. Questions?Web: http://people.kmi.open.ac.uk/roweEmail: m.c.rowe@open.ac.ukTwitter: @mattroweshowModelling and Analysis of User Behaviour in Online 34Communities

×