Fasten your seat-belt first & other safety reminders for Data Science & Machine Learning The age of Big Data, Internet of Things, Deep Learning, increasingly effective Cognitive & Artificial Intelligence, we seem to have technology & computational resources for unlimited data mining. Still, extracting valuable insights from data remains a real challenge. We'll tackle some of the typical issues & obstacles we face when trying to extract meaning & insights from observational data. The never ending data quality challenges, why correlation is not causation is just the start… Why it's so hard to extract real causes & factors from data, and... what is observational data anyway? Things you'll be glad to know before presenting your first data project insights.

1. Machine Learning
safety reminders
Data Science PT #7
rui.quintino@devscope.net

2. Where are we?
/groups/datascienceportugal
/datascienceportugal
@datascience_pt
2
/groups/8586496
/datascienceportugal ~ 320
/DataSciencePortugal
~ 278
~ 134
~ 110

3. And a special thanks to...
3
http://shelf.ai/
https://devscope.net/
https://www.uniplaces.com/

4. Data R&D @ DevScope
#PowerBI #SQLServer #Web
#Analytics #Azure #Microsoft
#MachineLearning #R #Linux
#Bots #Hadoop #Docker
#Python #Coaching #Learning
twitter.com/rquintino
rquintino.wordpress.com
rui.quintino@devscope.net
“jack of all trades (and master of none)“
1. a person who can do many different types
of work but who is not (necessarily…) very
competent at any of them…

5. DevScope

7. http://travelbi.turismodeportugal.pt/

8. https://www.sns.gov.pt/profissional/recursos-humanos-da-saude/informacao-rh/analise-mensal-do-balanco-social/

9. http://www.domussocial.pt/domussocial/caracterizacao-sociodemografica

10. C.H. São João- Reagir a tempo
Resultados
Financeiros positivos, Topo dos rankings nacionais
MSHUG Innovation Awards 2014
IT Europa’s BigData, BI & Analytics Solution of the Year 2014
Outstanding ICT Innovation Achievement HIMSS Europe 2016
Mais informação:
https://devscope.wordpress.com/2016/12/06/hvital-awarded-at-
himss-europe-2016/
https://devscope.wordpress.com/2014/04/02/iteuropas-best-big-
data-business-intelligence-and-analytics-solution-of-the-year/
https://devscope.wordpress.com/2014/02/24/hsjoao-devscope-
winners-in-the-microsoft-health-users-group-innovation-awards-
2014/
Centro Hospitalar de São João
www.hvital.com

11. TVI24 Eleições Media Capital - TVI
https://news.microsoft.com/pt-pt/2015/09/29/legislativas-2015-com-informacao-em-tempo-
real-numa-app-second-screen-criada-para-a-tvi24/

12. software

13. http://invoice.smartdocumentor.net

14. No templates used

16. https://www.kaggle.com/c/data-science-bowl-2017

18. Machine Learning & Data
Science Gotchas/pitfalls

19. •
•
•
•
•
•
•
•
•
•
•
•
•
•

20. http://drewconway.com/zia/2013/3/26/the-data-science-venn-diagram
Yeap… pretty much
that’s me he’s talking
about...

21. http://drewconway.com/zia/2013/3/26/the-data-science-venn-diagram
Actually,
honestly
precisely
here…

22. •
•
•
•
•

23. •
•
•
•

24. •
http://www.tylervigen.com/spurious-correlations

25. https://drsimonj.svbtle.com/exploring-a-causal-relation-between-healthcare-investment-and-child-mortality-in-r

26. •
•
•
•
•
•

27. http://www.public.iastate.edu/~dnett/S401/nexpvsobs.pdf

28. http://www.public.iastate.edu/~dnett/S401/nexpvsobs.pdf

29. •
•

30. http://blog.revolutionanalytics.com/2015/11/fun-with-simpsons-paradox-simulating-confounders.html

31. http://blog.revolutionanalytics.com/2015/11/fun-with-simpsons-paradox-simulating-confounders.html

32. •
•
•
•
https://en.wikipedia.org/wiki/Simpson%27s_paradox

33. Smokers, can use this as argument
Non-Smokers, please use this 
Ps-Please note: this is simulated data
https://app.powerbi.com/view?r=eyJrIjoiMjk1N2JmNWMtZmZhNS00N2EzLWEzYTgtN2YxMzExNTYzZjlhIiwidCI6
IjA5ZTI1MWRjLTVlODctNDhiZi1iNGQyLTcxYjAxYWRiOTg0YSIsImMiOjh9

34. https://app.powerbi.com/view?r=eyJrIjoiMjk1N2JmNWMtZmZhNS00N2EzLWEzYTgtN2YxMzExNTYzZjlhIiwidCI6

35. https://www.refsmmat.com/posts/2016-05-08-simpsons-paradox-berkeley.html

36. https://www.refsmmat.com/posts/2016-05-08-simpsons-paradox-berkeley.html

37. Ex: Simpson Paradox using Fisher’s Iris Dataset

38. •
•

39. https://www.perceptualedge.com/articles/visual_business_intelligence/variation_and_its_discontents.pdf

40. https://www.perceptualedge.com/articles/visual_business_intelligence/variation_and_its_discontents.pdf

41. •
•
•
•
•
•
•
•

42. •
•
https://www.kaggle.com/wiki/Leakage

43. •
•
•

44. http://nautil.us/issue/40/learning/is-artificial-intelligence-permanently-inscrutable

45. http://nautil.us/issue/40/learning/is-artificial-intelligence-permanently-inscrutable

46. https://homes.cs.washington.edu/~marcotcr/blog/lime/

47. •
•
https://xkcd.com/882/

48. https://xkcd.com/882/

49. https://xkcd.com/882/

50. https://xkcd.com/882/

51. http://www.kdnuggets.com/2015/07/deep-learning-adversarial-examples-misconceptions.html
http://karpathy.github.io/2015/03/30/breaking-convnets/

52. •
•
• https://betterhumans.coach.me/cognitive-bias-cheat-sheet-55a472476b18
• http://mentalfloss.com/article/68705/20-cognitive-biases-affect-your-decisions
•
• Occam’s Razor Hickam’s dictum
•
• https://www.youtube.com/watch?v=BrK7X_XlGB8

53. http://web.mit.edu/persci/people/adelson/checkershadow_illusion.html

54. https://www.youtube.com/watch?v=tleeC-KlsKA

56. • Pedro Domingos-A Few Useful Things to Know about Machine Learning
• Claudia Perlich - Leakage in Data Mining: Formulation, Detection, and
Avoidance
• Machine Learning Mastery- Common Pitfalls In Machine Learning
Projects
• Daniel Nee- Common Pitfalls in Machine Learning

57. •
•
•
rui.quintino@devscope.net

58. © 2016 DevScope. All rights reserved.
Rua Passos Manuel Nº 223 – 4º Andar
4000-385 Porto
T. +351 223 751 350/51
F. +351 223 751 352
Av. Sidónio Pais, Nº 2 – 3º Andar
1050-214 5 Lisboa
info@devscope.net
www.devscope.net