If you're applying for the role of a SDET (software design engineer in test), STE (software test engineer) or QA Engineer (quality assurance engineer), then the interviewer will invariably ask you to test "something". Recently, I've spent some time brainstorming a few testing interview questions with a few friends who are applying for a QA role. Based on my personal experience both as an interviewer and an interviewee, here's how I would answer a typical "testing" question:
How would you test a book?
Functionality:A book can be categorized functionally into two primary categories:
- Book to Read
- Test if the printed words on the cover and inside the book are legible and in accordance with the specifications on every page.
- Book to write
- Test if all pages have enough friction, so that it can be easily written/drawn onto them with a regular writing device.
- Check to see if the papers are of the right quality
- Check to see if the papers are of right thickness
- Check to see if the paper’s transparency is not too much so that one can read the book without seeing the words in the other page.
- Test if the book can be held in hand or set atop a tabletop, in such a manner that the user can open it and flip through pages.
- Check to see if the pages tend to stick together at any point
- Check to see if any of the pages are sticking out. This could be caused due to incorrect cutting machines
- Test, if the binding of pages and the cover is strong enough to hold the pages together, but flexible enough for the user to freely fold the book as per convenience.
- Make sure the pages do not have any obnoxious smell. The book should smell new, if it is coming out of a printing press
- Test for the acidity of pages, and ensure that it complies with regulatory standards.
- Check to see if the printing has caused any ink smudges
- Check to see if the printing has caused any aliasing of characters
- Check the weight of the book. For some grade school text books lower weight is a premium concern.
- Check the physical dimensions of the book like length, width, thickness etc.
- Test if the index section (a.k.a. chapter listing) is present and whether it’s matching to the specification, if given.
- Test if the book has a glossary of important terms
- Flip through the pages to see if the indexing of the chapter page numbers corresponds to the actual chapter beginnings. This is the main functionality of a chapter listing and hence this test is very important.
- Ensure that the content is appropriate for the target users. For e.g. ensure that there is no adult content in a children's book.
- Now start going through the tables and figures in the book making sure the bottom numbers are corresponding to the format and are in the correct order
- Next ensure that the fonts used are the same as the specification. If the testing time/resources is limited take random sample pages (try to get one from every chapter if possible)
- Make sure the figures and tables come in the right page where the matter is supposed to be
- Make sure that the margins are at right angle to the bottom of the book. This is to see if the entire text section is slanted after the printing
- Make sure the fonts are consistent (or according to the standard) throughout the book
- First of check and see if the title and outer cover is what it’s supposed to be. This can be accomplished by cross-checking with a given specification. Particular things to test are the author’s name, title name and making sure the graphics are what they are supposed to be.
- In the back cover check and see if the ISBN number is correct.
- Ensure that the barcode representation matches. You will need to know how to decode barcodes or use a barcode scanner which outputs the corresponding text string.
- Test if the price of the book is printed in proper format and as per specification. If the price is printed on both the inner cover of the book and the back of the book, ensure that both prices match.
- Move to the publisher information section to see if the dates correspond to what we expect. This is a good time to see if the edition of the book is correct. Here is where the copyright legal information is present and it is VERY important to make sure that it is indeed what the legal advisors mentioned in the specification
- Next is the most time consuming part. Reading the contents and verifying that the subject matter corresponds to the subject and the target audience. This is best done by a person well versed in the subject.
- Test for spelling and grammatical errors.
- Check to see if the author is able to articulate the ideas correctly, as per the user specification
- Test if the book contains a list of references, if the author has referenced some other works in the book.
- Test if the book has proper resources listed for user assistance. These may include but not be limited to author's contact information, if the reader has any concerns.
- The book should also have information about the publisher, if the user wants to order more copies of the same book or want to file a complaint.
- If the book contains a CD-ROM, test to see if the CD-ROM is readable in the computer environment given and that the data inside matches what is expected.
- If the book has an accompanying web site, ensure that the basic information about the book matches on both the web site and the book.
- If the book is a sequel/prequel to another book, then its contents should seamlessly correspond with the contents of its corollary.
- If the book has multiple-language versions, ensure that the printing press has not inserted the same contents of a page in a different language, maybe due to an error in the automated equipment
- Test for cultural differences in content of the book, so that it might not offend anyone in its target market
- If an international copy of the book has price written in more than one currency, then ensure that the price matches the specification
- If the book is published in special languages like Arabic/Persian, then ensure that the text is printed to be read from right to left and the index on the left side is wide-enough to facilitate seamless reading of the book.
- For a foreign language book check with a person knowledgeable in the script, to see if the characters are rendered correctly
Threat modeling and mitigation
- Use chemical tests to verify that the paper & ink does not contain toxic elements
- Check to see if the paper has the right amount of recycled material, either by any chemical tests or by verifying manufacturing records
- Check to see if the paper will make “paper cuts” to a user who moves his/her fingers along the paper edge. This can be accomplished by using wax and running the edge along it measuring the depth of the mark caused, instead of injuring the tester’s fingers!
- Take the book and throw it on the floor a few times to see if the bindings start giving up.
- If that is passed, try flipping through the pages as fast as you can, as many times as you can. This is the physical activity that will be performed most on a book and a good book should stand this test. If the quality of the book is not good, papers will start ripping off or at least get torn at the edges. This can be automated by using a machine similar to a bank teller’s currency counting machine to flip through the pages
- Test that the pages are not be torn when a certain amount of force is applied. Check this by trying to simulate that force
- If the book is a paperback (as opposed to a hardcover) try rolling the book and see how much permanent damage is done. It can be quantified using the curvature of the book after folding as a unit for measurement.
- If the above tests are passed, take the book through some testing of “the elements”. This includes high temperature to see if plastic coatings start melting before the temperature in the specs is reached, humidity to see if the paper quality gets eroded
- It is worth noticing some special cases at this point. If the book is to be used as a manual in a chemical factory, try to recreate the environment there (e.g.: chemicals in the air) and see if the book withstands it. Another example might be submerging the book in water for some time to see any physical changes, if it is supposed to be used as a map for white water rafters! A good test for computer books is to see how it can withstand coffee spills, since many software engineers prefer to have coffee to keep them going through late night sessions!
Book to write/draw:
- If the book has lines, then they are equidistant from each other and all pages have a margin of same size.
- Ensure that the pages have enough friction and are thick enough that, if the user erases and redraws, the paper is not torn apart.
- Ensure that the appointment book meets the functional requirements of having necessary tools like a calendar, and writing space for every date and time-slot.
- Ensure that the dates on calendar are correct
- Test that the pages are thick enough, so that they don't get damaged, if the user applies glue to paste pictures, etc on the pages.
- Test that the aesthetic appearance of the book appeals to the targeted user base.
Book to read:
- First of all confirm that the book can be unzipped using the recommended utility if it comes as a zipped file.
- Next make sure the book can be opened using the software it is supposed to be used with (e.g.: Microsoft Word, Adobe Acrobat Reader)
- Now use up the memory in the system and try to open the book at this low memory conditions.
- Scroll through the pages very fast and note how long it takes before graphic intensive pages become fully rendered.
- Make sure all the pages are contained in the book i.e. the file is not partial.
- Repeat these tests for various environments (e.g.: Windows, Unix, Apple etc)
- Repeat similar tests for Kindle (completely separate set of test cases)
- Test if the content is appropriate and appealing to a typical targeted child. Ensure that the book contains enough pictures and colors, as opposed to the regular black and white pages.
- Test for performance, as younger children are more likely to chew the book or try tearing away the pages more than adults.
Classified government/defense book
- Ensure that the cover of the book clearly states "Confidential" or "Classified" or any other appropriate word on cover, to ensure that the user understands its importance.
- Test if the book covers the functional and technical specifications of the product, for which it is going to be used as a manual
- Test if the book has labels, with the call-reference-id, so that it can be easily spotted on the shelves.
- Test if the book displays the name of the library
- Test if the book displays information regarding, when it is due back and the pertinent late-fees
- Hard cover
- Test that the cover-flap on the hard-cover has the correct information about the book, including but not limited to name, author, price, etc.
- For non-hardcover books, stack it vertically next to other books in a shelf to see if the shape degrades. This is a very common way for people to store books which makes this test relevant
Music Notes book
- Test that the pages do not make enough noise while flipping, because the music notes books may be used during live-recording of music
Manuscripts and antique books for museum
- Test if the pages are chemically treated to prevent any environmental damage to the book
- Test if the physical dimensions of the book are small enough to fit in a regular pocket.
- Test if the font-size is at least legible
- Very large folio-sized books
- Test if their size is justifiable
- Test if the size serves its purpose
- Test if the atlas is bound in such a manner, that if two adjacent pages form a single map, they can be completely opened flat to view complete details of the mapReligious books
- Test that the religious books are printed and bound in accordance with the beliefs of that religion. For e.g. A book on Hinduism should not be bound in cow's leather
Braille book for blind
- Test if the contents of the book are easy to find and comprehend by the visually impaired users
- Test if the size of the fonts, thickness of the pages and other physical dimensions of the book meet the compliance standards for Braille
- Test if the products and prices match
- Ensure that the user can find ordering information on the catalog
- Usually cloth books are written in form of scrolls, with wooden handles on both sides. Test that scrolls are arranged in the right order
- Test that the format of the audio file is in one of the popular standards and can be listened to in one of the popular audio device
If there's more interest on this topic, I'll follow up with a detailed post on testing interviews (how to apply for and how to recruit for awesome QA roles).
Let's break things!