UI Agnostic Tests
Date Author Description
10/16/2003 Nate Talbert (ntalbert) First draft
There are several design decisions which must be made in order to make it feasible for most tests to be run with or without working through the UI. WinOE’s Activity DOM certainly makes
this design easier, but it still does not answer all the questions. This document outlines the major decisions that must be made and proposes some solutions.
; The majority of tests should run in both “UI” and “non-UI” mode.
; While some restrictions will be necessary, there should not be too many requirements for
a test to be run in both modes.
; Test cases should be as simple as to write as strictly “non-UI” tests and should take care
of the extra UI overhead behind the scenes.
; Explicit UI test cases should be writable which exercise all of the UI, just as explicit non-
UI test cases should be writable which do API level testing.
This document is concerned mostly with design time UI agnostic testing. Down the road there will be other aspects of the WinOE testing that can be done programmatically or through a UI, but those design decisions will have to be made when these issues have more clarity.
Example Design Time Test Case Scenario
1) Create a new schedule
2) Add a send port
3) Compile the schedule
The test case code might look something like this:
ScheduleDescriptor sd = FAL.CreateNewSchedule("myTestSched");
AssemblyDescriptor ad = FAL.Compile(sd);
Non-UI View of the Scenario
From the non-UI side, this requires that CreateNewSchedule() maps to the following at the Activity DOM level:
ScheduleDecl sched = new ScheduleDecl(name);
Additionally, the ScheduleDescriptor need only contain a pointer to the ScheduleDecl that was created to make it a useful reference. The other two calls into the Functional Action Layer only require knowledge of the schedule’s root ScheduleDecl object in order to complete their actions.
UI View of the Scenario
The UI side, however, requires that CreateNewSchedule() performs the following actions:
1) Start the editing program
2) Create a new “project”
3) Create a blank schedule in the project
Additionally, in order to be useful information, the ScheduleDescriptor needs to keep track of the Editor Instance in which it is being edited along with the name of the file associated with the schedule. This should be enough to fully describe the schedule in the UI world and the other two calls should not need to know anything else about a UI-based schedule in order to complete their actions.
NOTE: In this scheme there will be only one project per VS.NET instance and only one schedule per project. That is, if another call to CreateNewSchedule() is made then a new instance of the editor will be started with a new project and a new schedule. This makes sense because the non-UI schedule declarations would also be completely independent. This has the advantage of not requiring the ScheduleDescriptor to have any knowledge of the VS.NET project/solution hierarchy, yet the disadvantage of requiring the creation of strictly UI tests to make sure that the project system gets fully exercises with multiple schedules per project and multiple projects per solution.
Design Solution 1
Create a class called ScheduleDescriptor which two other classes, UIScheduleDescriptor and NonUIScheduleDescriptor, will use as their base. Each of the subclasses will contain the appropriate information to describe a schedule for the corresponding design time mode. At the FAL level, most methods will look as follows:
UIScheduleDescriptor usd = (UIScheduleDescriptor)sd;
// perform UI actions using usd
NonUIScheduleDescriptor nusd = (NonUIScheduleDescriptor)sd;
// perform Non-UI actions using nusd
Each appropriate descriptor will require both UI and NonUI versions.
This has the advantage of being simple to implement and not requiring too much extra code on the part of the Functional Action Layer while still solving the complete problem. Also notice that if the UI mode gets changed somehow during test case execution an InvalidCastException will be thrown alerting the tester to the problem. Finally, this keeps the UI/non-UI abstraction fully contained in the Functional Action Layer which may or may not be an advantage.
Design Solution 2
Create the wrapper for the Activity DOM’s ScheduleDecl as a subclass with a class Schedule as its
base. A second subclass called UISchedule will be created from the base class Schedule. While ScheduleDecl will be a simple and straight forward wrapper, UISchedule will be required to keep track of such things as the editor instance in which it is loaded as well as the file which contains its data.
The code in FAL methods would look something like:
UISchedule us = (UISchedule)sd.sched;
// perform UI actions using us
ScheduleDecl schedDecl = (ScheduleDecl)sd.sched;
// perform Non-UI actions using schedDecl
This solution seems to have nothing but disadvantages. It spans both the Design Time Abstraction Layer and the Functional Action Layer in scope. It also requires the subclassing of decently complicated base classes.
Strictly UI Tests in this Scheme
Below is an example test that requires the UI and exercises the some more of the UI functionality.
SolutionDescriptor sd = FAL.CreateNewSolution("mySolution");
ProjectDescriptor pd1 = FAL.AddProject(sd, "Proj1");
ProjectDescriptor pd1 = FAL.AddProject(sd, "Proj2");
ScheduleDescriptor sched1 = FAL.CreateNewSchedule(pd1, "myTestSched1");
ScheduleDescriptor sched1 = FAL.CreateNewSchedule(pd1, "myTestSched2");
// This compiles the entire solution.
// This compiles just this project.
// Depending on implementation this could
// compile the entire solution, its containing
// project, or might attempt to compile the
// file containing this schedule in isolation from
// the project/solution.
Through overloaded methods like CreateNewSchedule() it is possible to make the strictly UI test case look almost identical to the agnostic test case. Also, by giving meaning to Compile() at so many different levels of the project scheme it is possible to test only what is really desired and allow the rest of the UI work to be done behind the scenes.
Handling Strictly UI/Strictly Non-UI Tests in the Wrong Mode
There should be an InvalidUIModeException which is thrown by the FAL when a mismatch occurs between the current Environment’s mode and the mode that the called method requires. When this gets to the test case level it should be caught and wrapped in a special subclass of TestCaseException called TestCaseModeMismatchException (or something similar). This way the failure can be logged as a test failure revealing a lot of information before investigation has even been started.