From Pilot to Product
•Download as PPTX, PDF•
3 likes•1,336 views
A sincere story about the successful project.
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (19)
Similar to From Pilot to Product
Similar to From Pilot to Product (20)
Recently uploaded
Recently uploaded (20)
From Pilot to Product
- 1. From PilotToProduct By Ivan Verhun
- 2. Notice • Ask questions during presentation • Time is limited • Business domain is not important
- 3. Agenda • Core product overview • Pilot architecture • • Dev Process
- 4. CoreProduct Overview • B2B • Import data from clients • Process data – Validate – Filter – Generate report
- 5. Data Flow
- 7. Main Disadvantages • Single tenant • Tightly coupled components – impossible to reuse • Business logic in stored procedures
- 8. This slide intentionally left blank
- 10. Pilot Overview • Target audience - SMB • Business pilot • Limited TTL • Preferably short TTM • Isolated from the core platform
- 11. Individuals
- 12. Businesses
- 13. Pilot Overview • Target audience • Business pilot • Limited TTL • Preferably short TTM • Isolated from the core platform
- 14. This slide intentionally left blank
- 15. Pilot Architecture • 1000 meters view • Technologies and Tools • Mistakes
- 16. Initial Key Decisions • Multitenant • Extract data from single tenant DBs • Functionality spread between modules • All business logic in Java – No stored procedures
- 17. Multitenant
- 18. Initial Key Decisions • Multitenant • All business logic in Java – No stored procedures • Extract data from single tenant DBs • Functionality spread between modules
- 19. Extract data from single tenant DBs
- 21. Endpoints • Asynchronous – AMQP endpoints for prod, qa, dev • Synchronous – HTTP endpoints for IT • Endpoints switched on/off depending on spring profile
- 22. This slide intentionally left blank
- 23. No Downtime Deployment • Maintain an application mode – LIVE – MAINTAINCE • Application mode is used by LB • Switch mode to MAINTAINCE before updating the application • Switch mode to LIVE after re- deployment
- 25. This slide intentionally left blank
- 26. TechStack– Before Tools • Java 6 • Jenkins • Maven 3 • MySQL 5.1 • Jetty 6 • RabbitMQ • Git/Github Libs and Frameworks • Spring – IOC – AMQP – MVC – JDBC Template – Aspects • JUnit • slf4j • Mockito, Powermock • Joda Time
- 27. Database Schema Migrations • Startup • Patch Level (patch0001_test_patch.sql) • Patch Loading • Patch Ordering • Patch Execution • Post-Patch Tasks
- 28. Newest DB Schema • Shell script generating DB dump • Runs on Jenkins • Commits newest dump to git repo • Used to create empty test DBs
- 30. BDDandJBehave BDD: • is an evolution of test-driven development (TDD) and acceptance-test driven design… • shifts the vocabulary from being test-based to behaviour-based • Was developed as a response to the issues encountered teaching test-driven development Author: Dan North http://en.wikipedia.org/wiki/Behavior-driven_development
- 32. This slide intentionally left blank
- 33. CentralizedLogging A tool for managing events and logs. You can use it to • collect logs • parse them • and store them for later use (like, for searching) http://logstash.net/
- 34. Kibana– ES data visualization engine
- 35. PerformanceMonitoring • Reporting Via JMX, HTTP, slf4j, CSV, Graphite etc • The Registry • Counters • Meters • Histograms • Timer (@Timed) http://metrics.codahale.com/
- 36. Java7 • Java 6 is dead • Java 8 is baby
- 37. This slide intentionally left blank
- 38. Persistence • JDBC was not OK • JPA/Hibernate was not enough • MyBatis – good for complex queries
- 40. TSVGeneration
- 41. ReportPrinter
- 42. PDF Validation Impossible to: • reliably verify absence of an element • associate present text to particular report attribute – there may be false positives • verify correct order of the text elements • parse/verify graphs http://pdfbox.apache.org/ Limitations Verify: • number of missing files • number of unexpected files • file content Scope
- 44. TechStack–After Initial tool kit • Java 6 • Jenkins • Maven 3 • MySQL 5.1 • Jetty 6 • RabbitMQ Added/updated tools • Autopatch • Configuration Service • JBehave + Thucydides • Java 7 • Logstash • Metrics library • Amazon S3
- 45. TechStack–After Original tech stack • Spring – IOC – AMQP – MVC – JDBC Template – Aspects • JUnit • slf4j • Mockito, Powermock • Joda Time Added later • JPA/Hibernate • MyBatis • Freemarker • Thymeleaf • Js: – AngularJS – RequireJS – Bower • Apache PDFBox
- 46. Mistakes • JDBC code is poorly maintainable • JUnit – @Parameterized tests
- 48. DevelopmentProcess • Methodology • Techniques and Tools • Mistakes and Lessons
- 49. Scrum Rules Principle Is Adopted Product Owner + Development Team + Scrum Master +/- Sprint planning meeting + Daily Scrum meeting + Sprint Review + Sprint Retrospective + Product backlog + Sprint backlog + Burndown chart -
- 50. SprintStructure
- 51. Planningprocess • Preplanning (30-60 min) – high level plans and priorities • Grooming (up to 3 hours) – discussion – estimation – Poker – Story points (0-8) • Planning (30-60 min) – final estimation – defining sprint scope (commitment)
- 52. Scrum Board
- 53. Task Workflow • Analysis and writing AC • AC Review • Coding – Unit tests – Functionality – Integration tests – Writing resolution comments • Code review • Verification on QA env
- 54. Daily Scrum
- 55. Daily Scrum
- 56. Daily Scrum
- 57. Daily scrum
- 59. Demo
- 60. Retrospective • What is working • What’s not working • To improve next time
- 61. CodeReviewTypes Ad hoc Passaround Pair programming Walkthrough Team review Inspection
- 62. Code ReviewTools – Github • Pros – inline comments – no extra software required • Cons – emails per comment (it's even not configurable) – inline diffs, no side-by-side comparison – small diff context – directory structure is not visible – Side-by-side comparison (Beyond compare) • lines comparison in the bottom (above status bar) • compare different files • compare with clipboard content – Reviewer checklist
- 63. BeyondCompare
- 64. BeyondCompare
- 65. ReviewChecklist • The Obvious – Code compiles, it’s clear, test cases etc. • Design • Coding Standards • Error Handling • Security • Thread Safety • Control Structures • Functions • Performance • Bug Fixes • Math • Maintainability • Documentation • Requirements Tracing
- 66. This slide intentionally left blank
- 67. PeopleManagement • Let people choose the task they like (if possible) • Avoid knowledge islands • Let people be initiative • Minimize micromanagement • Semi-periodic face to face meetings
- 68. Peoplemanagement • Dealing with highly motivated/enthusiastic people • Trainings – Agile (scrum) – Code review – Thucydides – Virtualization – TDD and Refactoring • Dinners
- 69. TeamBuildings
- 70. Thursdays Dinner
- 72. TeamComposition Ichak Adizes: • Producers • Administrators • Entrepreneurs • Integrators
- 73. This slide intentionally left blank
- 74. Mistakes
- 78. Code Review(ers) • Start with seniors only • Establish code review process before extending reviewers list • Extend reviewers list gradually • “Code Reviewer” title should be honorable
- 79. Learnings • Testers are essential • Sometimes micromanagement is a must • Establish code reviews right
- 80. This slide intentionally left blank
- 82. Why Customer NeedReportsTrust LowHigh Low High Transparency
- 83. This slide intentionally left blank
- 84. From PilotToProduct • Successful production launch • Unexpectedly big demand • Extended contract • Rewriting on core platform – New set of exciting technologies
- 85. Resources SaaS Maturity Model: • http://www.avuetech.com/wp-content/uploads/sites/7/2012/03/SaaS-Maturity-Levels.pdf • http://blogs.msdn.com/b/architectsrule/archive/2008/08/18/saas-maturity-model- according-to-forrester.aspx • http://www.infoq.com/news/2008/02/saas-architecture-maturity-model Code review: • http://en.wikipedia.org/wiki/Code_review#Types • http://www.softwaretestinggenius.com/istqb-foundation-level-exam-crash-course-part-6 • https://www.owasp.org/images/2/2e/OWASP_Code_Review_Guide-V1_1.pdf • http://smartbear.com/SmartBear/media/pdfs/WP-CC-11-Best-Practices-of-Peer-Code- Review.pdf Cross functional team • http://en.wikipedia.org/wiki/Cross-functional_team Customer management • http://habrahabr.ru/company/stratoplan/blog/202028/
- 86. Picture Sources • http://en.wikipedia.org/wiki/File:American_truck.JPG • http://www.expresstaxiinc.us/ • http://www.dreamstime.com/ • http://www.durhamcharterservices.com/ • http://www.galingas.com.ua/ • http://provse.te.ua/ • http://www.huffingtonpost.com/ • https://en.wikipedia.org/wiki/Scrum_(software_development)#mediaviewer/File:Scrum_process.svg
Editor's Notes
- Explain the SMB term Explain difference between Individual and SMB products Gas stations
- Gas station example Tips – pick up few passengers
- Similar businesses Business segment Schedule Length of the route Tips – pick up few passengers – doesn’t work
- Explain “Business pilot” term Reasons for isolation Level of isolation
- Change slide template
- SMB services: web app running on Jetty. Every service contains “similar” functionality There are 2 instances of a service Started with smb-dataimport and smb-reportgeneration services
- Multitenancy with multiple app instances (SaaS Level 4) Pros Support costs Ability to trigger a job for multiple clients in a single click Cons Integration with lot’s of single-tenant data sources Data isolation between clients
- Rule engine Separate business logic from system logic Created custom RE not based on JSR94 (JavaTM Rule Engine API) Used in Eligibility Pre-report checks Profile completeness %
- Connects to multiple Legacy DBs for data extraction Just a restart is needed to add new DB (no build required) Used org.springframework…AbstractRoutingDataSource Abstract {@link javax.sql.DataSource} implementation that routes {@link #getConnection()} calls to one of various target DataSources based on a lookup key. Created custom BeanFactoryPostProcessor to dynamically determine list of DBs
- Explain purpose of every file TSV file generation: Third parties data providers TSV Report data file Freearker vs Velocity vs Tiles
- Verify Seed files How PDFs are validated? Limitations Generally impossible to verify absence of an element There may be false positives – PDF may contain a text (e.g. month name) in some place, but this is not the place we expect Impossible to associate present text to particular report attribute Impossible to verify correct order of the text elements Impossible to parse/verify graphs PDF generator for integration testing No need to wait until printer provides generated seed reports
- What is Angular Client side MVC framework How it works reads in HTML that contains additional custom tag attributes; it then obeys the directives in those custom attributes, and binds input or output parts of the page to a model represented by standard JavaScript variables Angular teaches the browser new syntax through a construct we call directives. Examples include: Data binding, as in {{}}. DOM control structures for repeating/hiding DOM fragments. Support for forms and form validation. Attaching code-behind to DOM elements. Grouping of HTML into reusable components. Why Angular
- Tell why we have chosen every technology AMQP load balancing Consuming Stargate messages Spring MVC: to make it easier to trigger a process, we used HTTP endpoints to send an AMQP message later, we added a “synchronous” implementation of the HTTP endpoints for IT JDBC Initially used everywhere Now, used only in “Reading” part of legacy data extraction Aspects Refresh properties Tracking job start/end time, input parameters and status
- SQLs testing Dev Tools testing
- Few sentences about process: Scrum etc.
- Scrum master – removing impediments ensures that the Scrum process is used as intended.
- How branching is dictated by code review? Why do we use forks Types of code review
- https://wiki/display/PD/Code+Review+Checklist
- Team lead is between two fires – customers on one side and motivated people on the other Goal: ensure the customer is happy and people do what they like to do.
- Team lead is between two fires – customers on one side and motivated people on the other Goal: ensure the customer is happy and people do what they like to do.
- Dinners
- A cross-functional team is a group of people with different functional expertise working toward a common goal
- Іцхак Калдерон Адізес — один з провідних світових експертів в області підвищення ефективності ведення бізнесу і урядової діяльності шляхом внесення кардинальних змін, які не породжують хаосу і деструктивних конфліктів, що перешкоджають змінам. producers - those who work, like developers administrators - those who introduce processes entrepreneurs - new ideas, lets change everything - disruptions integrators - communicate, social glue no 4 qualities can be present, you should find others jobs - entrepreneur,
- We don’t need testers Sometimes micromanagement is a must
- Only senior and motivated person should review the code at first stage Establish code review process before extending reviewers list Extend reviewers list gradually “Code reviewer” title should be honorable