10 question business asked me
•Download as PPTX, PDF•
1 like•855 views
Доклад Андрея Мудрого на конференции SQA Days-17, 29-30 мая 2015 г., Минск www.sqadays.com
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (17)
Similar to 10 question business asked me
Similar to 10 question business asked me (20)
More from SQALab
More from SQALab (20)
Recently uploaded
Recently uploaded (20)
10 question business asked me
- 1. by Andriy Mudryy Competence Lead / Delivery Manager Remit AB 10 questions I’ve been asked as test Manager by CEO
- 2. Who am I • Hired by USA/SWE/RSA company to build a Dev center in Ukraine • Worked there for 2 years as All in one: • Development Center director • Delivery Manager • R&D Director • QA Manager (until hired a FTE for QA Mgmt.) • Agile Coach • Was a proxy between delivery and ExCo • When settled things up made a new opportunity in the company • Now working as Competence Manager & new customers Delivery Mgr. / CTO
- 3. Question 1 Now Andriy, when we hired an Agile-guru and a QA Manager (2 in 1)!.. … tell us how Agile will improve quality?
- 4. Executive Council has set the goal of 3000 Story Points / release • What would you need to help us achieve this? • How can we make QA to stop failing this goal – entering more and more bugs? Question 2
- 5. Waiting till end of sprint to add a feature we need? You should have some Agile tricks? Question 3
- 6. Aren’t developers supposed to write right? Why paying for QE? Question 4
- 7. Do we really need AQA to be Agile? Question 5
- 8. Agile looks to have lots of complicated rules. Can we KISS? Question 6
- 9. Testing brought some low hanging fruits. 850 reported! Looks like it’s worst release ever. Is next one going to be better? Question 7
- 10. Sometimes when I do something, something crashes. I wonder why that crash was not found. Question 8
- 11. We don’t want crystal clear quality. We have signed some ass kicking contracts. Our priority is time to market. What can I do for you to make it happen? Question 9
- 12. - Just finished reading some Retro stuff. - Q: What would you need to do better? A: “More whiskey” - WTF? Question 10
- 13. Now your 10 questions
Editor's Notes
- Hi guys! My name is Andriy Mudryy and I this is going to be a story about: Building a development center in Ukraine from the scratch Bringing the Delivery organization from 4 engineers sitting in the wardrobe to literary in the wardrobe so small the room for engineers was: well on the other hand the devs could turn on the light – but they decided the would be on the dark side of the Moon / Force Just to make things clear: it’s both and idiom and a real case: once when server was down and after fix was falling due to memory leaks and there was a session using a product a guy would sit behind the curtains listen to the ideas said and posting them with direct SQLs injections to the DB And what’s more interesting for you – that’s I speak here – is building QA process from the scratch. And I mean it. But before I start – let me give you a little bit of background: The company and product are really ass kicking. The product itself is a brainstorming and facilitating tool that changed the world so many times that I couldn’t believe: control economy of 2 continents, prevent a Nuclear War, send shuttles to space, react on H1N5 flu etc. Now the top 5 worlds most influential organization use this to build their strategies. I couldn’t believe that it could be done in such conditions.
- Now a bit of a background of what I was hired for and what I ended up doing… This is important because of 2 things: Who talks to you? And why I think I can talk about this? What was expected from me A USA/SWE/RSA company decided to extend the development team opening the office in Ukraine. I was hired primarily to make that office happen. The requirements to developers were really high: rock starts only. But… I got the budget for that. Anyway, the original plan was to hire 10 Java+JS Senior+ guys. Lviv was a good place for that 25%+ of the IT workforce in Ukraine. But there were hardly 10 guys at all. And other 100500 companies would let them go easy. And there’s was no one to care about the quality. So just stated that (those guys just like when you DARE to CARE to tell truth straight ahead in the face) that there is no chance to do this even with given money. So I suggested an alternate solution to hire some QA (since no quality at all on the product), Automation, DBA etc. Still freaking rock-stars but to fulfill the team. Like in football – you cant have a team of 10 forwards and the goalkeeper. And those guys in the Top mgmt. like metaphors – we would talk about them later on as well. So that’s how I became the Delivery director and a QA manager (at lest till the time I hired one). Also I as I was discovering the company and learning it needs it occurred that: There was just the Agile philosophy (true Agile philosophy – need to say – but nothing besides that) – so became an Agile expert There were cool engineers with innovative ideas : needing a leader – so I became an R&D Director So I ended up with 6 hats (almost like in Walt Disney creative strategy) This speech will be focused on QA most – but some other “hats” will happen too. I don’t work with the project any longer – so now I can talk more :) I set the things up and running and saw no much new challenges So I started looking a building challenges for myself (a good experience – just go and say what you want to do in the company) but still this was one of the most thrilling experiences in my life.
- Now, Who of you have faced this question? Like being agile is more or less quality… First thing we need to make sure that quality is quality disregard the process you work. You need to ensure the quality is the quality you plan. It’s just needs to satisfy the quality baselines. Here’s one more analogy I use speaking with business using their language. ThinkTank changed the Business Model from being a donator organization to earning selling the services to World most economical most influential organizations: DTT, EY, PWC, Accenture. And as I’ve been told by the CEO of the company – those guys drive BMW. And they need BMW quality everywhere. The truth was – that the quality of product we were selling to them was mot better than 1960-s hippie VW mini-van. And VW is definitely what we wanted to sell. Speaking about the Agile culture… It was in the air. But nothing else. No working frameworks, no rule – just a good example how Agile could be chaotic. You need to include quality in the mindset and the process. But since business believed in Agile and me I just recommended few simple steps: Test Cases Performing Testing User Stories acceptance Definition of Done (the last two makes even more sense in general, since Quality comes qualitatively only then when you have it your mindset). You can’t do something BMW style if you day to day act as Band of Blustery Baboons in the old garage. Or well… wardrobe
- And how about this question? Check out those fancy business phrases: “What would you need to help us achieve this” – we gonna achieve, you gonna do. And who is the main enemy? Right, QA they keep finding bugs all the time. And that’s why we don’t move quickly. Looks familiar? Now my answer was: Story Points – are non-dimensional value. Gummi-bears basically. Measuring distance with ball-throws. A relative value. Business does not understand story points good. The main outcome of my whole speech comes at this slide: Business understands when you talk money If you say this thing costs 3000 money? Is it much? You never know. Well… unless it’s a computer game. But even in computer games it differs. The same here 3000 US Dollars is not 3000 Indian Rupees. You need 3000 SP – I can re-shale the SP now and you get 30 000. But the team wouldn’t do more software just because that If you keep doing slack-slack and let it go into production – this won’t make a VW to BMW just if you place the sign there.
- Who of you have ever faced such suggestions? Much, right…. This was another coaching moment for business. And yes, business needs some coaching even they pay you money. Actually they pay you as a manager so that you coach them. It’s like one of those Steve Jobs quotes that they post on LinkedIn with his face on it. Or with some grumpy cat face. Whatever… You hire a person so that he says what to do, not does what you say. So… Agile does not mean Scrum. And half of IT companies can’t tell one from the other. Don’t like Sprints? Don’t like the retro’s & stuff? You got Kanban. Then you prioritize things quicker. But everything has it’s trade-offs. Then don’t require long term detailed plans. If want to prioritize just do it. If you want the dates – then do some planning. By this I mean that ability to improve is more important than sticking to the plan. But this does not mean you don’t plan at all. Actually this is all we did – we had a team of firemen that were working just on fires. Not planning. Pure KanBan – but still some rules. They (I mean complications) even led to kissing. So stay tuned!... Where we ended up is creating a team of team of the new feature developers working with Scrum and the SWAT team working on some hot things sold to be done yesterday. Or to help support to save the world… once again. But guys working in SWAT team need to rest some time so we organized rotations for both Dev and QA. Test design development was done only in Scrum Teams – and KanBan team was only running the selected suits.
- QE stands here for quality engineers. This is possible but very unrealistic. Especially if developers had worked on the product for couple of years. Now with that question we returned to what we started. Have you ever seen developer testing it’s own code? Except probably the unit tests. Could you imagine a developer willing to find a bug or an edge case in his perfects and elegant code? Not likely. In the very end – if you state that you want BMW quality you should admit that: Haters hate Potatoes potate Developers develop Analysts analyze And testers test. The same here – if you want something get done professionally, you better hire a professional. This does not mean QA can’t help developers find issues early in the process. What we did were: Developer Test logs – like checklists that developers need to check during testing Peer reviews – after the Test logs were checked QA just sat with Devs to try simple break scenarios so that Developer could fix that ad hoc
- Well to be Agile you need to think Agile and act Agile. AQA will help you save money on being Agile. And deliver more frequently with better quality. Let’s get back to what we already discussed here: the most effective way to talk to business is to talk money. When we structured the test cases – it occurred that we have 5300+ test cases documented. (and about 4700 not documented) Just running the test cases (not taking into consideration bug posting and verification) took 9 man/weeks. Since new versions were released every 2 sprints – this required a team of 6 junior developers be dedicated on testing and running regression tests. 6 Juniors * month = 3K USD just on stupid monkey clicking. This does not include actual testing of the new functionality, new tests, security and scalability testing. 1 senior AQA hired can build a framework and automate (which was proven) in 15 months. ROI – 15 months. This is just salary-wise and not talking about the operational expenses. Profit – money. Even more important is time to market perspective. And we will talk more on that that during question #9. For the team of 6 developers 9 man/weeks is 10-12 calendar days. For and automation running thru the night on couple of devices this is 1 night. Profit 9-11 calendar days.
- What would you do if someone from the Board of Directors asks you something like this? What if it was a lady? A real-lady business shark? KEEP CALM & LEARN ACRONYMS KISS = Keep it Simple Stupid. Why those question occurred at all? Because when ensuring more quality and building the process always leads to some NO-s that needs to get said to business owners if they want to break the rules. I got the two parts of the response: Now imagine that BMW says we don’t want to follow the rules and deliver the new car to the market without checking/re-checking/double-ckecking. In the majority of cases this leads to calling back the whole series of cars and crisis-management. Do you lack fire-fighting and crisis-management now? And yes, you got my KISS – you do the business and don’t care about the process and the rules. I will let you know when you want to break them. You hired rock-stars. So them rock and KISS for you.
- One of the classics! Who is responsible for so many bugs found in the project? Right, QA. When we started Project Measurement (Metrics) activities we started identifying how much bugs are there now open in release. When there was no QA – this number was about 100-150. When QA started looking into this application and listing everything in 2 releases this number grew to 850? So for the first point of view – more QA more bugs. More complication. And that could lead for the whole QA team to get fired. And this is real. What I learned from this – metrics a good. Your feedback on them, analysis and recommendation is better. And this how a trust you as a QA Manager is built. Your predictions and forecasts come true. The release next to the one with 850 bugs – became the best release for last 5 years as per customer feedback.
- What is the correct answer to that? Your options…. What I did – is I closed the bug with my comment “something fixed”. But this is just a real example of how things could work. We created an e-mail, where users could post some feedback on the application. Obviously 80% of the feedback was notifications about bugs. Some of them was like that. And investigating those was real pain in the neck. Like what was the reason. Especially for those schroeding-bugs that’s reproduced once in full blue moon. And fixing that was a night mare. But here are the benefits that we taken from that: Users were willing to help We showed them that we CARE This was just a kicking point when that emails had attachments with what version is used and what is the system state Even if we couldn’t repro those bugs – these emails were used to build a map of problematic places (or the places users use most in day to day work)
- Check out that Management trick once again: What can I do for you to make it happen? The most important here is that business admitted – I always thought. That the time to market is more important than quality. Which was obvious in the times when there was no quality at all. But this is a typical case, too. First you are told that time to market is a priority. Then you get asked why this did bot got caught during testing. Well because the time to market did not give enough time to test all the workflows and edge cases. Test cases are just atomic touches. So what I told – is asked to put the name and signature under the statement. Made a nice frame and put the well-printed PDF certificate of this quote. Well, this is not only CYA letter, this just change my mindset a bit from being to orthodox. Life happens and you need to be ready for that. Now when suggesting options – I don’t take quality as a constant. This could be a price business sometimes is ready to pay.
- And finally. Now couple of words about the QA itself not the QC. We tend to call QA just the testing, which is QC – Quality control. QA is the whole another thing. It’s the quality of the processes that lead to deliverables in the project. Long story short: If you want to get qualitative answers – you need to ask qualitative questions. The Retros were just ceremonial. And questions were asked just to get asked. And it happened that CEO was checking the notes. So this also needed to get redesigned. But this is a whole another story…
- Now I you can ask me some questions ))