1.0 Item Development Tick all ID Functional Requirement Description I want this ID 1.001 Can add an MCQ question ID 1.002 Can add an EMQ question with unlimited stems ID 1.003 Can add an Objective Structured Clinical Exam (OSCE) question ID 1.004 Can add short answer question ID 1.005 Can add essay questions ID 1.006 Can add multi choice multi answer questions ID 1.007 Can add drag and drop questions ID 1.008 Can add hot spot questions ID 1.009 Can add an Objective Structured Video Exam (OSVE) question ID 1.01 Can add a missing word question ID 1.011 Can retrieve an MCQ question ID 1.012 Can retrieve an EMQ question ID 1.013 Can retrieve an OSCE question ID 1.014 Can retrieve a short answer question ID 1.015 Can retrieve an essay question ID 1.016 Can retrieve multi choice, multi answer question ID 1.017 Can retrieve drag and drop question ID 1.018 Can retrieve hot spot question ID 1.019 Can retrieve an OSVE question ID 1.02 Can retrieve a missing word question ID 1.021 Can modify an MCQ question e.g. can change descriptive metadata of question ID 1.022 Can modify an EMQ question e.g. can remove a stem of question ID 1.023 Can modify an OSCE question ID 1.024 Can modify a short answer question ID 1.025 Can modify an essay question ID 1.026 Can modify a multi choice, multi answer question ID 1.027 Can modify a drag and drop question ID 1.028 Can modify a hot spot question ID 1.029 Can modify an OSVE question ID 1.03 Can modify a missing word question ID 1.031 Can delete an MCQ question ID 1.032 Can delete an EMQ question ID 1.033 Can delete an OSCE question ID 1.034 Can delete a short answer question ID 1.035 Can delete an essay question ID 1.036 Can delete a multi choice, multi answer question ID 1.037 Can delete a drag and drop question ID 1.038 Can delete a hot spot question ID 1.039 Can delete an OSVE question ID 1.04 Can delete a missing word question ID 1.041 Can import an IMS QTI MCQ question ID 1.042 Can import an IMS QTI EMQ question ID 1.043 Can automatically alphabetise answer options ID 1.044 Can assign multiple curricular classifications to a question ID 1.045 Can add/remove searchable tags/descriptors/keywords to any assessment item ID 1.046 Can add one question author ID 1.047 Can retrieve one question author ID 1.048 Can modify one question author ID 1.049 Can retrieve all authors ID 1.05 Can mail merge authors to email and letter ID 1.051 Can add one question writing workshop ID 1.052 Can retrieve one question writing workshop ID 1.053 Can modify one question writing workshop ID 1.054 Can delete one question writing workshop ID 1.055 Can produce automated time dependant workshop alerts ID 1.056 Can produce thank you letters/certificates ID 1.057 Can link questions to flexible blueprint ID 1.058 Can link questions to author, source of question, date etc ID 1.059 Can view questions already in bank by blueprint ID 1.06 Can have theme and focus ID 1.061 Can store who inputs questions and when ID 1.062 Can force certain fields to be completed when entering questions ID 1.063 Can have pictures and symbols ID 1.064 Can use formatting e.g superscript, subscript ID 1.065 Can develop and store items in a secure, encrypted environment ID 1.066 Can have multiple users accessing system at once ID 1.067 Can have item development via web and offline ID 1.068 Can detect duplication if items added ID 1.069 Can store images with questions ID 1.07 Can store sounds with questions ID 1.071 Can store videos with questions ID 1.072 Can import questions from MS Access ID 1.073 Can import questions from MS Excel ID 1.074 Can import questions from Speedwell ID 1.075 Can import questions from SQL Server ID 1.076 Can interface with electronic dictionaries ID 1.077 Can enforce date/time stamps on all items input to system ID 1.078 Can over-write date/time stamp for items imported to system ID 1.079 Can specify a date for item review e.g. 2/5 years from date of creation ID 1.08 Can specify clustering/bad pair rules upon item creation ID 1.081 Can support standard setting and can add/remove standard setting method e.g. Angoff 2.0 Quality Assurance ID Functional Requirement Description I want this QA 2.001 Can automatically generate a report that displays the current level of questions prepared for review QA 2.002 Can perform task as in QA.001 but excluding questions created at one institution QA 2.003 Can allow different stages of quality assurance QA 2.004 Can provide a full audit trail QA 2.005 Can provide full version control QA 2.006 Can interrogate audit trail (e.g. produce report to compare items pre-review/post-pre-view) to highlight pre/post QA. QA 2.007 Can create a new ID for a new version of item if item has been locked-down and user wants to change it QA 2.008 Can track where an item is in exam preparation at any time QA 2.009 Can allow individual questions to be marked `ready for next stage of QA` QA 2.01 Can allow QA to follow a set path i.e. Gold Standard Test QA 2.011 Can add reviewer directions to questions QA 2.012 Can force reviewers and other users to complete one or more minimum actions on questions QA 2.013 Can prevent different users from different actions on questions QA 2.014 Can add comments visible to different users with different access rights QA 2.015 Can allow reviewer users to pass a decision on questions, adding queries QA 2.016 Can display total no. of questions, questions in development/ready QA 2.017 Can choose multiple search parameters to find and isolate questions for review QA 2.018 Can move questions to module (internal or external e.g MS Access) to review and can import these questions back recording each as having completed appropriate stage of QA QA 2.019 Can have multiple (endless) stages of review and can track where questions are at any time QA 2.02 Can have more than one way of progressing through QA i.e not a straight line, questions can be brought back a step and can leave normal course of events and then come back. All this can be recorded. QA 2.021 Can have choice of online or offline QA, with different users preferences on each occasion QA 2.022 Can have multiple QA end points and reasons recorded (flexible drop down lists) QA 2.023 Can search for any word in any area of question or metadata QA 2.024 Can prompt for next stage of QA dependant on previous events and schedules QA 2.025 Can name stages of QA QA 2.026 Can choose and record who is responsible for items being reviewed/put in QA QA 2.027 Can lock-down items to prevent changes to item e.g. they`re due to appear/have appeared in an exam QA 2.028 Can record whether author has been trained to write items 3.0 Item Storage and Selection ID Functional Requirement Description I want this IS 3.001 Can add one exam IS 3.002 Can retrieve one exam IS 3.003 Can modify one exam IS 3.004 Can delete one exam IS 3.005 Can automatically exclude questions for use at specific institutions based on past usage IS 3.006 Can manually select questions individually IS 3.007 Can automatically select questions against a flexible blueprint IS 3.008 Can automatically select questions against performance data IS 3.009 Can store `curriculum blueprints` for future use IS 3.01 Can flexibly define and choose parameters for question selection IS 3.011 Can save different methods of question selection (i.e different schools have different parameters that they use and re-use, making changes from time to time) IS 3.012 Can output selected items in variety of ways including to MS Word format IS 3.013 Can output selected items in variety of ways including to MS Access format IS 3.014 Can output selected items in variety of ways including to MS Excel format IS 3.015 Can output selected items in variety of ways including to XML format IS 3.016 Can see history of question selection episodes and track these forward, by user IS 3.017 Can link question selection instance to a future exam episode and a user (Medical School) IS 3.018 Can have several stages of item selection IS 3.019 Can have secure online selection of items by parameters IS 3.02 Can allow or disable online/offline ability to view entire question or other parts of a question/metadata IS 3.021 Can view summary via blueprint of items as they are being selected and at any time after selection complete IS 3.022 Can enforce selection time limits if necessary e.g. time that selection can take and time period when selection is possible IS 3.023 Can flag up items as clustered/bad pairs IS 3.024 Can enforce capture of info about an exam e.g. exam date/name, year group, institution/school, no of items, type of items IS 3.025 Can store EMQ stems/options as separate fields in order to facilitate searches IS 3.026 Can randomise ordering of questions of final exam layout IS 3.027 Can enforce use of anchor questions from one exam instance to the next IS 3.028 To test robustness IS 3.029 Solution capacity - Can store infinite questions and exams data IS 3.03 Solution security - Can have different levels of security access IS 3.031 Solution scalability 4.0 Exam Delivery ID Functional Requirement Description I want this ED 4.001 Can distribute the selected questions electronically for printing ED 4.002 Can distribute the selected questions electronically for delivery online ED 4.003 Can distribute the selected questions by paper for delivery ED 4.004 Can allow exam administrators to mark the questions as used within the system ED 4.005 Can distribute the selected questions for final checks by External Examiners ED 4.006 Can have separate internal module (or work with external module e.g. Access) which will retain details of use and link IDs to exam paper question numbers ED 4.007 Can integrate with existing systems - WebCT/Blackboard VLE ED 4.008 Can integrate with existing systems - Fronter VLE ED 4.009 Can integrate with existing systems - Teknical VLE ED 4.01 Can export to other item banking solutions - Questionmark ED 4.011 Can export to other item banking solutions - Speedwell ED 4.012 Can export to other item banking solutions - TRIADS ED 4.013 Can export to other item banking solutions - TOIA ED 4.014 Can output into IMS QTI v2.1 compliant format ED 4.015 Can output into IEEE LOM compliant format ED 4.016 Can output into SCORM compliant format 5.0 Results Analysis ID Functional Requirement Description I want this RA 5.001 Can import data from an excel spreadsheet of binary data RA 5.002 Can analyse question performance RA 5.003 Can analyse exam performance RA 5.004 Can analyse student performance RA 5.005 Can perform plagiarism detection RA 5.006 Can assign parameters to facilitate suspension of items RA 5.007 Can allow second QA pathway to address performance RA 5.008 Can store response data RA 5.009 Can perform item analysis and display basic stats RA 5.01 Can define formulas for item analysis/calculations: discrimination, facility, per response, per answer - can do all this for multiple year groups and multiple occasions, showing averages where appropriate RA 5.011 Can check `show all results for this questions facility` or `show average of all facility` RA 5.012 Can store and analyse by student registration number and look at how students progress over time, forecasting fails RA 5.013 Can select/change QA status of any item individually or as part of a group (multiple items) RA 5.014 Can show results per blueprint/curricular domain e.g. average ENT score 75% RA 5.015 Can display results by keyword/place of authorship e.g. flexible way of interrogating results RA 5.016 Can hold information about setting a standard score e.g. standard errors RA 5.017 Can interface with a statistics package e.g. SPSS RA 5.018 Can have flexible feedback routes for different purposes e.g. students/course planners/standards setters
