So you‘ve built an outstanding lesson using one of the Articulate tools, either Studio or Storyline, and it’s time to publish and upload it into your LMS. You’ve arrived at the publishing screen and have made your choice to publish as SCORM 1.2 or 2004, but are then asked to select how to “Report status to the LMS.” What’s the difference between all of the options and how will each affect what’s reported to your LMS? Is one option better than the others? Before we discuss what option you might select, let’s discuss how these values came about in SCORM 1.2 and 2004. Initially, in SCORM 1.2, there was only one Lesson Status element with possible values of “completed,” “incomplete,” “passed” and “failed”. It quickly became evident though that with a single value, it was difficult to accurately determine how the learner actually performed. When a learner exited a ten question quiz after only answering a few questions, most authoring tools would report “failed” rather than “incomplete.” Due to the limited information received, the LMS was unable to differentiate this scenario from a true failure; one where the learner actually completed the quiz and failed it.
The subsequent version of SCORM (2004 aka 1.3) corrected this problem by replacing the single status with two: Completion Status and Success Status. Complete Status would indicate the completion of the lesson as “completed” or “incomplete” while Success Status would indicate the learner’s performance as “passed” or “failed.” We could now differentiate between all possible scenarios.
Since SCORM 1.2, Articulate products have provided you with the same “Report status to LMS” option, but we realized that with the changes between the SCORM versions and the confusion we saw from our clients, it was not clear exactly what behavior would result from each choice. So for this paper we have created two types of lessons, those with a quiz and those without a quiz, and will examine the results for each of the various options.
Before going further, note that the values we show in the tables below are those that the lesson will provide the LMS and what the LMS will display unless they have been altered by functionality related to mastery score settings. Therefore, if you have a mastery score defined, these values will very likely be altered. Details of the mastery score logic can be found in the SCORM 1.2 and SCORM 2004 Run Time Environment documentation (SCORM 1.2: Section 3.4.4 and SCORM 2004: Section 4.2.22.1). Note that some of the options we select will cause Articulate to automatically insert a mastery score. These scenarios are noted in the tables below and explained further.
In the tables that follow, the “Completion Status,” “Success Status” and “Learner Sees” values listed will appear in the following places within the Learn by HireRoad LMS:
- Reports (using the Course Transcripts dataset)
- The “Completion Status” appears within the column labeled “Lesson Completion.”
- The “Success Status” appears within the column labeled “Lesson Success.”
- Learner’s Transcript (within their My Account)
- The “Completion Status” appears within the column labeled “Lesson Completion.”
- The “Success Status” appears within the column labeled “Lesson Success.”
- Learner’s “Launch” Page (within their My Account)
- The “Learner Sees” value will appear in the column labeled “Status.”
The following tables are separated into two categories: Lessons with Quizzes and without Quizzes. It is interesting to note that both Articulate Studio and Storyline had the same output for lessons with quizzes, there are subtle differences between the two products for lessons without quizzes. The differences are noted for each as well as our general recommendations for the best overall publish settings.
Lessons with Quizzes
For our test lesson with a quiz, we included a four question quiz and configured it with a passing score of 70%. Our results focused on each SCORM version and the following three distinct scenarios:
- The learner partially completes the lesson (but not enough to pass it) and then exits.
- The learner completes the lesson and passes the quiz (over 70%).
- The learner completes the lesson and fails the quiz (less than 70%).
Articulate Studio & Storyline
- With Quiz
- Published as SCORM 1.2
Setting: Completed/Incomplete
| Scenario | Completion Status | Success Status | Learner Sees |
|---|---|---|---|
| Partial Completion | incomplete | unknown | In Progress |
| Pass | unknown | passed A | Passed |
| Fail | incomplete | unknown | In Progress |
Setting: Completed/Failed
| Scenario | Completion Status | Success Status | Learner Sees |
|---|---|---|---|
| Partial Completion | incomplete | unknown | In Progress |
| Pass | unknown | passed A | Passed |
| Fail | unknown | failed | Failed |
Setting: Passed/Failed
| Scenario | Completion Status | Success Status | Learner Sees |
|---|---|---|---|
| Partial Completion | incomplete | unknown | In Progress |
| Pass | unknown | passed | Passed |
| Fail | unknown | failed | Failed |
Setting: Passed/Incomplete
| Scenario | Completion Status | Success Status | Learner Sees |
|---|---|---|---|
| Partial Completion | incomplete | unknown | In Progress |
| Pass | unknown | passed | Passed |
| Fail | incomplete | unknown | In Progress |
A Articulate sets a Mastery Score, causing the LMS to change your “completed” value to “passed”.
Articulate Studio and Storyline
- With Quiz
- Published as SCORM 2004 3rd Edition
Setting: Completed/Incomplete
| Scenario | Completion Status | Success Status | Learner Sees |
|---|---|---|---|
| Partial Completion | incomplete | unknown | In Progress |
| Pass | completed | unknown | Completed |
| Fail | incomplete | unknown | In Progress |
Setting: Completed/Failed
| Scenario | Completion Status | Success Status | Learner Sees |
|---|---|---|---|
| Partial Completion | incomplete | unknown | In Progress |
| Pass | completed | unknown | Completed |
| Fail | completed | failed | Failed |
Setting: Passed/Failed
| Scenario | Completion Status | Success Status | Learner Sees |
|---|---|---|---|
| Partial Completion | incomplete | unknown | In Progress |
| Pass | completed | passed | Passed |
| Fail | completed | failed | Failed |
Setting: Passed/Incomplete
| Scenario | Completion Status | Success Status | Learner Sees |
|---|---|---|---|
| Partial Completion | incomplete | unknown | In Progress |
| Pass | completed | passed | Passed |
| Fail | incomplete | unknown | In Progress |
Observations and Recommendations
It is interesting to note, that Articulate includes a Mastery Score in the manifest for all lessons that include quizzes published as SCORM 1.2. By doing so, the LMS will never show “completed”, even though you may have set Articulate to report that way, since the SCORM specification requires the LMS to override the success status if a mastery score is set.
If you are looking for an all-around guideline for publishing a lesson with a quiz, we recommend that you publish in SCORM 2004 and set the “Report status to the LMS” option to “Pass/Fail.” We’ve found this to be the most logical in reporting and will only provide an “unknown” value for the success status while the lesson is still in progress which makes sense (since the lesson’s success cannot yet be determined). If you complete the lesson, either pass or fail, you will get a logical status reported back to the LMS with no “unknown” status anywhere. If you don’t want your learners to ever fail a lesson, you may want to choose to publish in SCORM 2004 and set the “Report Status to the LMS” as “Passed/Incomplete.” This setting will show the user as “In Progress” for that lesson up until they actually pass the lesson.
Lessons without Quizzes
For our test lesson without a quiz, we included 5 screens, and configured it such that you had to view 4 of the 5 screens to complete the lesson. Our results focused on each SCORM version and the following two distinct scenarios:
- The learner partially completes the lesson (but does not view enough of the screens to complete it) and then exits.
- The learner completes the lesson by viewing 4 or more of the screens.
Articulate Studio & Storyline
- No Quiz
- Published as SCORM 1.2
Setting: Completed/Incomplete
| Scenario | Completion Status | Success Status | Learner Sees |
|---|---|---|---|
| Partial Completion | incomplete | unknown | In Progress |
| Full Completion | completed | unknown | Completed |
Setting: Completed/Failed (Storyline Only)A
| Scenario | Completion Status | Success Status | Learner Sees |
|---|---|---|---|
| Partial Completion | incomplete | unknown | In Progress |
| Full Completion | completed | unknown | Completed |
Setting: Passed/Failed
| Scenario | Completion Status | Success Status | Learner Sees |
|---|---|---|---|
| Partial Completion (Studio)B | unknown | failed | In Progress |
| Partial Completion (Storyline) | Incomplete | unknown | In Progress |
| Full Completion | unknown | passed | passed |
Setting: Passed/Incomplete
| Scenario | Completion Status | Success Status | Learner Sees |
|---|---|---|---|
| Partial Completion | incomplete | unknown | In Progress |
| Full Completion | unknown | passed | Passed |
A We have listed this setting as “Storyline Only” because we discovered a bug in Articulate Studio that causes the lesson to always report its status as “failed” regardless of whether the learner completed the lesson. Because of this, we recommend that you never use this setting in Articulate Studio when your lesson does not contain a quiz.
B When using Articulate Presenter (Studio) for a lesson that has no quiz, and publishing in SCORM 1.2 with the reporting set to “Passed/Failed”, we discovered an inconsistency; this setting, while logical in that it reports “failed” for a partial completion as the selection indicates, is unlike all the others that report “incomplete” to the LMS. Because of this, we recommend that you never use this setting in Articulate Studio when your lesson does not contain a quiz.
Articulate Studio & Storyline
- No Quiz
- Published as SCORM 2004 3rd Edition
Setting: Completed/Incomplete
| Scenario | Completion Status | Success Status | Learner Sees |
|---|---|---|---|
| Partial Completion | incomplete | unknown | In Progress |
| Full Completion | completed | unknown | completed |
Setting: Completed/Failed (Storyline Only)A
| Scenario | Completion Status | Success Status | Learner Sees |
|---|---|---|---|
| Partial Completion | incomplete | unknown | In Progress |
| Full Completion | completed | unknown | Completed |
Setting: Passed/Failed
| Scenario | Completion Status | Success Status | Learner Sees |
|---|---|---|---|
| Partial Completion (Studio)B | completed | failed | In Progress |
| Partial Completion (Storyline) | Incomplete | unknown | In Progress |
| Full Completion | completed | passed | passed |
Setting: Passed/Incomplete
| Scenario | Completion Status | Success Status | Learner Sees |
|---|---|---|---|
| Partial Completion | incomplete | unknown | In Progress |
| Full Completion | completed | passed | passed |
A See footnote ‘A’ in previous table.
B When using Articulate Presenter (Studio) for a lesson that has no quiz, and publishing in SCORM 2004 with the reporting set to “Passed/Failed”, we discovered an inconsistency; this setting, while logical in that it reports the Success Status as “failed” for a partial completion as the selection indicates, also reports the Completion Status as “completed” which is clearly not accurate. It is also unlike all the other options that report partial completions as “incomplete” and “unknown” to the LMS. Because of this, we recommend that you never use this setting in Articulate Studio when your lesson does not contain a quiz.
Observations and Recommendations
Since we found discrepancies between the reporting within Articulate Studio and Storyline and actual bugs in the Studio (Presenter) product, we really can’t make a single publish recommendation for both products when publishing a lesson that does not contain a quiz, but is measured by screen views. Generally, publishing in SCORM 2004 with the report settings set to “Passed/Incomplete” seems to make sense for both products, and displays a somewhat logical outcome. Normally, with a lesson that has no quiz, the user should never fail, but only complete the lesson. Most of the reporting preferences simply come down to a personal preference as you may not want to show a lesson with no quiz as being “passed”, but as “completed”. As with the lessons that included a quiz, we recommend that you publish in SCORM 2004. If you want a learner to view a successful completion as “completed”, use the “Completed/Incomplete” setting. If you want the user to view a successful completion as “passed”, then use the “Passed/Incomplete” setting.