Teaching Software Testing On-line

As part of the NSF-Funded project "Teaching Software Testing On-line," we are developing a series of four testing modules that can be used in several courses across an undergraduate computer science curriculum.

Each module will cover approximately 3 weeks of student time, although the content will allow the instructor to adjust the coverage assigned to students and assign content from the module in parallel with ongoing programming assignments. Rather than simply being a recorded lecture, the modules will each contain a written, cross-referenced presentation of testing concepts and material together with on-line, automatically graded self-tests, pre-written exercises that can be assigned for out-of-class work, and additional exercises that allow an instructor to incorporate his or her own material into the student's assignment. Each on-line module will also provide an on-line student feedback mechanism for students to comment on the module itself.

This work nicely complements the other CS education projects being conducted at Virginia Tech.

Four On-line Modules

The four modules being developed as part of this project are:

The interns working on this project should look at the current InternAssignments.

Delivery Strategy

Our on-line modules are designed to be highly interactive. They are set up to be delivered using an adaptive hypertext system called NetCoach. As the student works through a module, NetCoach builds an internal model of what knowledge the student has mastered so far. This model is based on the student's answers to exercises and self-test questions.

Based on an estimate what the student currently knows and where the student has been so far, NetCoach adaptively annotates the links to various sections of the course to personalize the navigation to the student and to subtlely advise the student on what material they are most ready to learn.

Module Structure

Each of the four modules has a similar structure, designed to take a student approximately three weeks of out-of-class participation to complete (in parallel with other course assignments). Each module is divided into three sections, punctuated by on-line quizzes or tests. Each of these sections is subdivided into a hierarchically structured group of 20-40 separate web pages, each of which focuses on one specific topic or concept.

To support NetCoach in building a model of what the student has mastered, each individual page has associated with it a small collection of 2-5 test questions that can be automatically graded on-line. This body of questions forms a pool from which tests can be constructed.

Each module follows this rough outline:

  1. Pre-test: On entry, the student takes a pre-test of about 20 questions. This allows NetCoach to assess the student's current knowledge and let them skip over material with which they already are familiar and jump right into the content they are ready to see. Questions are randomly drawn from the entire module.

  2. Section 1: The first of three major divisions of the module.

    • Self-test 1: One third of the way through the module the student will take a self-test of about 30 questions on-line. This test is designed to give the student feedback on how well the content in the first section of the module has been mastered. Questions are randomly drawn from the pool associated with the first third of the module. Although scores are available to instructors and TAs, this is primarily a self-assessment activity and students are allowed to attempt the self-test as many times as desired.

  3. Section 2: The second of three major divisions of the module.

    • Self-test 2: Two thirds of the way through the module the student will take a second self-test of about 30 questions on-line. This test is designed to give the student feedback on how well the content in the second section of the module has been mastered. Questions are randomly drawn from the pool associated with the middle third of the module. Although scores are available to instructors and TAs, this is primarily a self-assessment activity and students are allowed to attempt the self-test as many times as desired.

  4. Section 3: The third of three major divisions of the module.

    • Self-test 3: At the end of the third section of the module the student will take a third self-test of about 30 questions on-line. This test is designed to give the student feedback on how well the content in the third section of the module has been mastered. Questions are randomly drawn from the pool associated with the last third of the module. Although scores are available to instructors and TAs, this is primarily a self-assessment activity and students are allowed to attempt the self-test as many times as desired.

  5. Post-test: At the end of the module the student will take a final on-line test of about 50 questions. The questions are drawn from all across the module. This test is designed to assess the student's performance in mastering the materials. The student may only attempt the post-test once. Scores are available to instructors and TAs for use as desired.

NSF Sponsorship

We gratefully acknowledge the support provided to this work by our sponsors. This material is based upon work supported by the National Science Foundation under Grant No. DUE-0127225. The project grant falls under NSF's Course, Curriculum & Laboratory Improvement (CCLI) program, as a "proof of concept" project in the Educational Materials Development track. An abstract of the grant is available on-line. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.