Testing

6.1 Introduction

Some of the black box testing was already carried out during the implementation stage. Few of the exceptions that remained were handled after completion of the implementation process. Every effort was made to handle all possible exceptions. This testing was considered to be very important for the following usability tests for which smooth functioning of the application was essential. The details of black box and usability testing are described in this chapter.

6.2 Black-box testing

The black box testing is divided into two parts. First is a list of things that could go wrong on part of the user and hence to be caught as exceptions and handled accordingly. And second is a list of things that the application should do correctly when an action is performed. These lists should not be considered as complete since it is practically impossible to do testing in its entirety. Hence, some of the most important tests have been conducted.

List of exceptions tested and handled:

When a non XML File is opened.
When an XML file of different format is opened
When no file is opened and is double clicked on navigation viewport.
When no file is opened and mouse is double clicked on examination viewport.
When no file is opened and mouse is double clicked on images viewport.
When no file is opened and mouse is double clicked on text viewport.
When no file is opened and an attempt is made to find distance in navigation viewport.
When no file is opened and an attempt is made to see some or any of the categorized text.
When no file is opened and an attempt is made to see the thumbnails.
When no file is opened and an attempt is made to identify the clickable objects.
When no file is opened and an attempt is made to clear the examination viewport.
When no file is opened and an attempt is made to clear the text viewport.
When no file is opened and an attempt is made to clear the image viewport.
When no file is opened and an attempt is made to clear the all the three viewports.
When no file is opened and the image toggle is switched on.
When no file is opened and the text toggle is switched on.
An attempt is made to double click on an empty examination viewport.
An attempt is made to double click on an empty text viewport.
An attempt is made to double click on an empty image viewport.
An attempt is made to double click on a visual object, which is not clickable.

List of actions performed (only after a data file has been opened):

For the results of the following actions, please refer to appendix 3 User Manual.

Correct data file is opened.
Examination viewport is cleared by clicking the Clear Exam button on the tool bar or the menu bar.
Same as no.2 for Clear Text and Clear Image buttons.
All three viewports are cleared by clicking the Clear All button on the tool bar or the menu bar.
If any text is displayed in the text viewport and the text viewport is double clicked.
While a big text window is already open, the text in the text viewport is changed and then double clicked.
If any objects are displayed in the examination viewport and the object is double clicked.
If any image is displayed in the images viewport, the images viewport is double clicked.
While a big image window is already open, the image in the image viewport is changed and then double clicked.
When the left mouse button is dragged inside the examination viewport.
When the left mouse button is dragged while pressing the alt key inside the examination viewport.
When the right mouse button is dragged inside a non-empty examination viewport in the upward and downward direction.
When the left mouse button is double clicked in the Examination viewport while the mouse is over the detailed object.
When the arrow keys of the keyboard are used in the navigation viewport.
When the arrow keys of the keyboard are used in the navigation viewport while keeping the shift key pressed.
When the page up key of the keyboard are used in the navigation viewport.
When the page down key of the keyboard are used in the navigation viewport.
When the left and right arrow keys and the page-up and page-down keys of the keyboard are used in the navigation viewport while keeping the alt key pressed.
When the left mouse button is double clicked on a clickable object in the Navigation viewport.
When only the image toggle is switched on and user navigates in the navigation viewport.
When only the text toggle is switched on and user navigates in the navigation viewport.
When both the toggles are switched on and user navigates in the navigation viewport.
When the distance finding facility is turned on and two points are selected in the navigation viewport.
When the clickable objects finding facility is switched on and mouse is moved over any part of the navigation viewport.
When the categorized text button is clicked on the toolbar or the menu bar, some check boxes are selected and result button is clicked.
When the thumbnail images button is clicked on the toolbar or the menu bar and one of the images is clicked.
When the Manual button is clicked on the toolbar or the menu bar.
When the About button is clicked on the toolbar or the menu bar.
When the Contact button is clicked on the toolbar or the menu bar.
When the left mouse button is clicked once in the Navigation viewport and remains inside the viewport and then exits.
Same as 32 for Examination, Images and Text viewports.

6.3 Usability testing (Top)

An attempt was made to get participants from the target user group i.e. An architectural conservationist, a student of architecture/ archaeology specializing in heritage structures and an enthusiastic person interested in learning more about heritage (representing the general population). Due to time constraints, it was not possible to conduct usability tests with an architectural conservationist. The final participants were 1. a student from architecture, 2. a student from archaeology 3. a student from business management, and 4. a student from computer science, all four being post graduates in this University (names mentioned in acknowledgement section).

Before the usability tests, heuristic evaluation of the application was done in addition to preparation of justification for some known critical comments (related to problems in using virtual reality applications on PC platform).

Heuristic evaluation:

No irrelevant information is provided.
No system-oriented terms are used.
Users memory load is reduced by not changing functionality of commonly used functions. E.g. File open dialog, mouse left button for selection by double clicking etc. Appropriate colours were used to represent common metaphors e.g. Red for “Quit”, Green for “Start” or “ON” and neutral colours for other states of the buttons. Tooltip text was also provided to give immediate help to a novice or first time user. Keyboard arrow keys were used to implement navigation.
Wherever practical, consistency was followed.
Warning messages were provided before performing any clearing actions or before quitting the application. Appropriate feedback was also given when a process took long time (more than 10 seconds). Toolbar buttons provide feedback by changing colour when mouse is over a particular button. Each viewport shows its activated state by drawing a rectangle around itself when the mouse is clicked once in the viewport.
Exit is clearly marked.
Automatic display of images and text is provided as an alternative for people already having good experience of using the application. Furthermore, menu items can be accessed through standard keyboard shortcut key “alt” + mnemonic.
Polite language is used to describe any error messages (e.g. when a XML file of different format is opened).
Every attempt has been made to avoid all types of errors through black box testing.
User Manual is provided and is made easily available at the click of a button.
Bigger buttons are provided for easier and faster target acquisition (Fitt’s law).
Most interacted areas are located at bottom right or on the right side of the application and are placed together to allow faster movement between them (Fitt’s law).

Justification for critical comments:

1. It does not look real!

The tool should not be considered as a replacement for reality but more of an aid to find information quickly and easily. But, as the computational limitation at the PC level drops by the day, better looks along with better facilities will be a possibility in the near future. The database and the application are constantly under development and critical feedback from users is most welcome.

2. Some details are missing or not correct!

Some anomalies may exist in terms of details due to undocumented data or it may still be under progress but is made available nonetheless.

3. It is too slow!

The amount of data to be stored is enormous and every effort is made to make all types of data available in the correct form with correct references. To load this data for the first time takes some time. Compared to the time taken to visit various organizations to search and refer to different types of data, this time limit is quite insignificant. Hence, some patience will be required while using this application. A strong effort has been made to make the experience an enjoyable one, and also to make the application as transparent as possible.

A questionnaire was prepared for each tester (see appendix 2) and the results were evaluated. The following observations and conclusions were made:

All four students found the navigation facility very exciting and easy to use. There was some irritation and uncomfortability in some parts where the key presses sometimes stopped responding due to overload of processes. But this was found to be very insignificant compared to the excitement of being able to navigate in the virtual world.
Identifying of clickable objects was found to be a difficult concept. Since this particular facility is yet to be developed properly, no judgement is made.
Referring to and comparing between multiple text and images was found to be an important feature by the archaeology and architecture students. They felt that this was an important feature in the application for use by specialists. Similarly, the thumbnail images and the categorized text information windows were considered very useful. Referring to a single close up photograph was found to be a long task; hence this point was taken into consideration for further development.
Distance finding facility was one of the most difficult and confusing facility to use due to toggle behaviour of the button and also the method of its implementation along with integration with the other functions of navigation in the navigation viewport.
Opening a file, double clicking actions were quite naturally used, as they were standard and are commonly seen in all other applications. The users found it very easy to use this technique in examination of various parts of the structure.
As expected, some irritation was found in the loading time.
A strong comment was made about reliability of information (database) given by the application. In response, the user was shown the source of the database, and all its references, which were already mentioned in the application. This was an important point, since unreliable data would lead to distrust on the application causing a bad name. For a specialist, accurate information is of vital importance and hence every effort has been made to provide all references on the source.

Overall, it was found that the user needed more practice and familiarizing (atleast for two hours before testing) to use and test the full extent of the application. One and half hour of testing time was not adequate to make complete judgments for the software. Nonetheless, the users found the interface of the application, easy to use, if not very easy and that the application has a good future potential.

Previous Home Next