Let’s start by listing SCRUM’s key features and aspects for those unfamiliar with it:
Live Typing has employed SCRUM when working on several products to a certain degree of success: the team attained discipline without managerial supervision; the tasks were better assessed because their evaluation happened down the line when several pitfalls had already been detected, and the customer was able to provide prompt and thorough feedback. However, successful implementation of SCRUM left us wondering how we would incorporate the already existing testing process into the methodology, so now we share with you the results of our testing department’s experimentation with SCRUM.
"This case study is an apt illustration of how complex SCRUM implementation can be when the methodology is introduced into an existing work sequence, despite how easy it may sound in theory. Failed experiments, mistakes, misrepresentation, bureaucratic hurdles and other problems that pile up during SCRUM implementation are all a tricky obstacle course to conquer, and even though the folks from our testing department haven’t overcome it completely, they’re still making great progress. What counts is that they examine their results, come up with solutions, refine their methods and work hard on their attitude towards the product and their work in general with a shared objective in mind."
Alexey Pimenov, Agile Coach in ScrumTrek
Our company’s priority area is mobile app development. As a result, the working conditions are sometimes far from perfect, with tight deadlines, lack of documentation (often the case with startups) and customers who change their requirements frequently and suddenly. This pits our testing department against a whole range of challenges, with shortage of time being the primary concern. While a large enterprise project brings with it a huge amount of requirements, it’s easier to deal with it than with a mobile startup as it provides pre-approved documentation and the schedule is not too limited. The scale of mobile development projects is usually much smaller, but due to the nature of mobile platform testing, distinctive requirements for mobile apps and frequent absence of technical documentation, the workload is comparable with larger projects, while the amount of time dedicated to testing is cut short.
This leads to the following issues with mobile app testing:
Requirements change on the fly, defeating the purpose of TOR/technical specifications and making them a waste of valuable product development time.
Communication and awareness issues within the team are also unheard of. For instance, a developer might ask the customer about a certain detail and overlook to share it with the team, or a tester might miss a Scrum during which important decisions have been made (but not documented due to a lack of time), and so on. As a result, the team might lack a clear idea of the project. This translates into the next problem on our list:
The work process is often organized in such a way that a build is handed over to testing only when the majority of features are complete. A tester that hasn’t been following the product’s development closely will burn through a lot of time getting to know the project before starting to actually test it.
If your app supports (for example) calendar integration, uses custom-made chats, obtains and shares social network data or interacts with cloud services like Google Drive, testing difficulty will grow exponentially because of how diverse the app’s interactions are.
The main problem of implementing testing in SCRUM was the testing routine that was present at the time, which was formed with long development stages in mind, after which the build was to be handed over to testing. It involved bug report acceptance, corrections, more testing, more corrections and so forth until the features that had been planned for this stage were fully implemented. However, SCRUM is designed around short Sprints that each end in a complete and workable product increment. Eventually, we established two scenarios.
Scenario 1. For our purpose, we’ll call it “Single test + fix check”.
This way, the bugs found in a current Sprint were corrected only during the next. Nevertheless, because the customer was aware of the project status and the final acceptance was done only at the end of a stage, there was more time put aside for tests and fixes in the Sprint.
Advantages of Scenario 1:
This approach was very similar to the one that we used before the introduction of SCRUM, which allowed us to familiarize ourselves with the new methodology in a relatively safe way. The features were delivered to the customer fully tested by the end of a stage, leaving our hands (and conscience) clean.
Disadvantages of Scenario 1:
Let’s illustrate this drawback with an actual case. Our tester was used to asking the manager for all kinds of clarifications, but for the manager it was more convenient to examine the project on their own account and send the build to the customer as quickly as possible, telling the customer about possible flaws in the product, instead of handing it to the tester who could reject the build and delay the build shipping date. Communication issues went unresolved and resulted in bugs that the tester would learn about virtually on the last day of the project. On the contrary, SCRUM declares that every team member is equally and willingly responsible for the product. Having learned this lesson, the tester would directly consult the customer afterwards.
We did not fully comprehend the SCRUM principle and mindlessly followed the “no new tasks in the current Sprint” rule. We took a similar attitude to bug fixing, eliminating critical bugs only by the next Sprint, putting off low-priority errors for later, up to a point where we would have an entire Sprint dedicated entirely to bug fixing at the end of a stage.
Occasionally the customer would like to test the product on their own, and the team wouldn’t have a fixed list of the tasks that were supposed to be completed in a given Sprint, except for the list of tasks for the stage. This led to the following events:
Don’t shoot yourself in the foot like we did and make it a rule to clearly designate tasks for a Sprint and eliminate critical bugs in the current Sprint.
This scenario resulted in a quality product, but the work process was not entirely SCRUM. The customer was somewhat discontented to discover bugs on their own, but was satisfied with the final product.
Breathing in and out, running right along.
Scenario 2. For our purposes, we’ll call it “Test flow”.
This scenario was much closer to actual SCRUM, even though tasks were still sometimes added to Sprints. The duration of Sprints was also altered a few times.
Advantages of Scenario 2:
Productivity would indeed increase if all of these conditions were met. Every day at about the same time the tester received the builds and provided reports for the team to fix the critical issues if necessary. This allowed the tester to better plan their runtime and ensured that they wouldn’t be pulled out to work on a different project.
Disadvantages of Scenario 2:
There is another side to this scenario: if a non-critical, non-blocker bug was discovered, the task it pertained to was marked as “complete” anyway. It was supplemented with a comment that described the bug or had a link to the bug tracker attached, and the PO examined it and made the final decision to accept or reject the task. It was somewhat awkward to knowingly supply the customer with a buggy build.
This scenario turned out to be a better approach, though it wasn’t perfect. To smooth out the rough edges, we set a few additional rules for ourselves.
Time for your second wind to kick in – we’re entering the homestretch!
The only kind of task that you can and should add to a Sprint is fixing blocker bugs. It is acceptable to add other tasks if and only if the initial tasks are fully complete, 100% tested and the features are absolutely proven to work as they are supposed to.
Each of these builds has to be a product increment. If everything goes south after you add the latest feature and the solution threatens to be time-consuming and complicated, you can send the PO one of the previous builds, as at least one of them has a decent chance of being operational.
You may successfully test your build on Nexus 5, Galaxy S7 and Xperia Z3 only to hear a less-than-flattering response after your customer tries to run it on their LG G3. The mobile device market (especially if we’re talking about Android) is very diverse. It’s immensely difficult to fine-tune your product for every device, so make sure you agree upon which devices you cater to.
It is common for bugs to crawl out at the last possible moment, so don’t condemn yourself to working on weekends and carrying around a bunch of test devices off work. This is sure to take its toll on the quality of your testing, so make sure that your Sprints end on a Tuesday or a Wednesday so that the pre-delivery finishing touches are made as part of your routine.
We’ve also introduced a kanban-like element into our team’s work procedure. Sometimes, when the list of tasks for testing is growing, the developers won’t hand the build to the tester (because they want to complete as many tasks as possible before sending the build away); or there is a lot of tasks prepared for testing, but the tester won’t send the build over to the PO, wanting to present the PO with the largest possible part of the tested product. This has a negative impact on everyone: the tester has an emergency as there’s a lot of tasks to be tested but not much time until the end of the Sprint; the developers have an emergency as the tester has dumped an entire truckload of bugs at their doorstep and they also haven’t got much time until the Sprint ends; even the PO is in an emergency as the Sprint draws to its end and the team is still deep in it, fixing and testing. To prevent this, we decided that each team member should not have more than (for example) two tasks while the total amount of tasks that are tested or prepared for testing must not exceed (in this case) six if there are three developers on the team.
First and foremost, SCRUM helps if there is no documentation and facilitates communication within the team. The methodology helps clearly define and gradually assign the tasks and maintain ongoing contact with the customer’s representative, and generally makes it easier for everyone to get a grasp of the project. In the meanwhile, you can compile test documentation if you wish.
In SCRUM, testing begins right after development has started, which helps the tester to stay in the know from the very beginning. Abrupt changes in customer requirements are promptly discussed with the team and are accepted or rejected collectively because the PO stays in touch with the team and quickly gets the idea of what can and can’t be implemented.
You’ve crossed the finish line! While these methods can’t guarantee the successful completion of a project, they might at least help you implement SCRUM in your work practice. We hope you find our experience insightful.