On a daily basis, a group of brave warriors work seamlessly within the MindTouch echelon of Engineering and Development. We are the keepers of the “Wall” of MindTouch Software, we defend against the hoards of unexpected bugs, random interactions, and unmet expectations. We are constantly in training to make ourselves and our testing better, stronger, and faster. We are the QA Department.
The QA Department is an interesting part of the Software Engineering Team. If the QA Engineers are doing our job well, no one outside of the Engineering Department hears about it. The changes and updates to the software are smooth and seamless.
From the outside, all features are working as expected, and the new features are working as designed. The previous features and the newest updates are working seamlessly together, without integration issues.
Within the department, exciting skirmishes pop up every day. We are on alert for any unusual interactions, ways of using the product, combinations of keystrokes, any of which could show discrepancies. With each new check-in, the QA Engineering Team is putting the MindTouch software through its paces.
However, we have secret weapons. We use more than the speed and accuracy of our own eyes and hands to verify behavior. We have two separate suites of automation tests. One suite is the Screenshot testing, discussed here, by Zach : https://mindtouch.com/engineering-our-success/screenshot-testing-mindtouch/
The second test suite automates different user experience functions: clicking buttons, filling in forms, verifying errors, verifying permissions, user interaction and settings. It tests user interactions completed within the browser.
Selenium Webdriver Automation Tests. (Insert choir sound here!!!)
These two suites of tests, Screenshot testing and Selenium Webdriver tests, run completely separate from the build process. They are both manually triggered at any time, and often are triggered at key points in the release cycle. Between our manual testing and automation testing, the QA Department verifies that a huge number of features work and look as expected. In the event that a test fails, a report is generated for us to investigate further. We can assess quickly if further effort outside of the QA team is needed, or if we need to update the test to keep pace with development.
I joined MindTouch in October 2016. During my training phase, I manually tested the MindTouch software following a specific group of tests. It helped me learn the product and interact with the team.
Then Dan introduced me to the Automation Suite. Most of the manual tests are automated, within the realms of what is possible for Selenium Webdriver. A test which took me 10 minutes or longer to run manually, including complex setups for user accounts and permissions, takes 20 seconds to run on my machine. It’s amazing!!!
The QA Department is currently automating a backlog of tickets. These are bugs that have been fixed, some of them were bugs seen or reported on customer sites. We are developing tests to verify the fix, and once the test is checked into the Automation Test Suite, every single time the test is run, we make sure that particular issue has not resurfaced!!!
Recently, I listened to an orientation taught by one of the founders of MindTouch, Aaron Fulkerson. He shared the history and company values of MindTouch. “Satori” is a concept of complete understanding and comprehension, and MindTouch seeks to increase Satori for every person who uses it. He showed us a graph of users and their learning curve. Once they reach a high level of understanding, they become masters and teachers of that concept.
Within the QA Department, we have varying levels of understanding in writing automation tests. We spend a lot of time teaching and reviewing one another’s code. The more time we spend seeking Satori, the more stable our test suite becomes over time.
Automation, Screenshot, and Manual Test Suites are used in conjunction to assess the overall health of the newest developments within MindTouch, as well as troubleshoot issues and send valid feedback to the rest of the Engineering Department. Each method has its strengths, and each method has proved invaluable.
The QA Team is committed to “Satori.” We have many plans for the future of our testing. Together, we’re making our testing better, faster, and stronger, along with the entire MindTouch application.
[Sarah Burgess has been with MindTouch since October 2016, and is a strong advocate of automation testing, superhero powers and music.]