Over the past week or so I have sent over some content to Litmos so they can show off the beta of their new mobile app that can play and track Articulate content on mobile devices that support the Flash Player.  Well last night Litmos founder @richchetwynd posted the video below showing Articulate Quizmaker working on his Google Nexus One.

This is awesome and is making me want to get a Nexus One  Check it out:

Mobile SCORM tracking of Flash based content on the Google NexusOne phone from Litmos Team on Vimeo.

So you know you want an LMS.  Good. This article isn’t here to tell you why you need one, or why you should use one, this is a guide for anyone who already knows they want an LMS, just wants a guide for what to check for.

Over the years I have spent many hours testing content and trying various different Learning Management Systems, and have even done some LMS (like) design work with Articulate Online.  Over that time period I have had the opportunity to learn a lot about what does work well, and what doesn’t work well in a lot of systems, so based on my knowledge on the subject, here is my list 10 things to consider before choosing in an LMS.

1. What LMS Standard does it use (AICC, SCORM 1.2, or SCORM 2004)?

There are 3 LMS standards of significance these days, AICC, SCORM 1.2 and SCORM 2004.

A little history on the standards…

The AICC standard was developed by the aviation industry in 1993 and has evolved a bit over the years, but remains pretty similar to its roots.

The SCORM 1.2 standard was released in 2001 and is the most popular standard (in terms of usage) to date, and was actually developed by the department of defense.  It was at least in part designed based on some of the limitations of the AICC standard.  Although some consider to be an improvement over AICC, it still had many shortcomings.

SCORM 2004 was released in 2004 (shocker huh?) and in this specification they addressed some of the shortcommings and ambiguities of SCORM 1.2.

In the years that I have worked with Learning Management Systems issues have come up where the LMS expects one thing, but the content expects another.  A big reason this has happened is that the early standards like SCORM 1.2 have left a lot open to interpretation.  For example, certain versions of Saba will expressly ignore the status that the content sends to the LMS if the content exits with a “suspend” mode.  With our content we expressly exit with a “suspend” mode when a user exits a course, which will normally allow a user to relaunch the content.  So when you launch and complete content in Saba we expressly tell the content that the status is “completed”, but Saba ignores this because the status is set to “suspend”.  Both sides are technically doing what is right, but still we fail.  Find a solution to the Saba problem here.

There are other places where the earlier standards were just flat out missing something that should have been included.  In the SCORM 1.2 standard there is no option for the content to communicate with the Learning Management System what the question was.  It allows you to send an “interaction id”, but this is generally assumed to be an ID, and not question text.  So the standard allows the content to communicate what type of question it was (t/f, MC, MR, etc), what the correct response was, and what the learner response was, but no way for us to communicate what the question was.  In SCORM 2004 they improved this part of the standard by allowing us to pass the question “description” which allows us to communicate the question text.

Some of the older standards have some shortcommings based on the age of the standard.  Back when AICC and SCORM 1.2 were developed, we weren’t in the same place that we are today in terms of computer processing power, bandwidth, etc.  This caused the standards to be limited in what they allowed the content to send.  For example, the SCORM 1.2 standard only allows content to send 256 characters to the LMS for a learner response to a question.   So, if you have an essay question that you ask your users, any response they give over 256 characters would violate the SCORM 1.2 standard.  Again, this is something that has been improved in SCORM 2004, where they raised the limit to around 4000 characters.

So as the standards have evolved, generally they have been improved along the way, with each newer version  addressing shortcomings, ambiguities and limitations of the previous veresions.  So make sure you consider what standard or standards (many support more than one) the LMS supports before making your purchase decision.  If your LMS supports more than one, or all, GREAT!

Other things to note when thinking about standards:

• Most major content authoring tools support AICC, SCORM 1.2 and SCORM 2004
• If you often use custom developed content, make sure your content developer also can work with the standard of your LMS choice
• Beware of vendors that only partially implement a standard.  Both the ADL (who controls the SCORM standard) and the Aviation Industry for CBT Committee offer certification for LMS vendors.  If an LMS claims AICC, SCORM 1.2 and SCORM 2004 support, see if they have been certified as compliant.

2. What technologies are required by the LMS?

Ok, I admit it, I have had a love hate relationship with Java…the hate part stems for a Java class I took in college and 2 years I spent working at Sun Microsystems.  The love part is that Java can do some really cool things cross platform (i.e. Screenr).

Java definitely has its place, but LMS environments is not one of them.  See post and comments here for more info.

Every once in awhile I get a report that our content doesn’t work in some LMS running a specific version of Java (example 1, example 2).  Why it works in every browser, and non-Java based LMS, but has issues with specific Java versions is a serious pain in the ass to figure out.

I could rant for another 2 pages on everything I think is wrong with Java in a LMS environment, but that is not the point of the article.  The point is, look at what is required by the LMS to view content.  Does the LMS work without any additional plug-ins or odd system requirements?  Also check to make sure your LMS system requirements match the systems your users will be using.  Do they support Safari browser on a Mac, do they support Firefox?  Do they support IE6 and later?

Note: JavaScript is different from Java.  JavaScript is cool, Java is not.

3. How easy is it to get content up on the LMS?

One of the problems with many Learning Management Systems is that getting content up to the LMS takes a monumental effort.  I kid you not, in some Learning Management Versions it is a 15 step process of getting content up to the LMS.  Seeing that getting content up to an LMS was often a difficult task, the AICC board along with several eLearning experts developed the PENS standard (Packet Exchange Notification System).  The idea was that the content authoring tool like Articulate Presenter or Captivate would notify the LMS that some content was available, and the LMS would then go pick up the content from the authoring tool.  This sounds great, and is great, but the problem is that most content vendors and Learning Management Systems are slow to adopt new standards.   It is kinda like the problem with clean fuels for cars, gas stations won’t get green fuel until cars start using them.  PENS is an improvement, but it hasn’t been adopted by most LMS vendors or content vendors, so it is important to see how easy it is to upload content to the LMS.

Use Scripted Demos

Map out common things that you are going to need to do with the LMS, and ask your LMS Sales Representative to walk you through those tasks.

Some ideas for scripted demos:

• Upload content to the LMS
• Enroll a student
• Run reports on tracked content

These are day to day tasks that you are going to want to do, so you should know how easy or how difficult this will be before purchasing.

As a bonus, ask your sales rep to take content you have and upload it to the LMS and prove that it works.

4. Focus on ease of use of the features you need, not on the number of features the LMS offers.

We use to have a bugbase that we used at Articulate that was truely aweful.  It was slow to load, took forever to log a bug, had poor text handling and would always throw exceptions, it was just a terrible bugbase for actually logging and tracking bugs, and those should be the core features.  Every year the bugbase would release a new version and they would keep piling on new features that I didn’t care about (CRM Management, Case Management, external bug reporting system), without making the core features work.  The company that made the bugbase was about competing on the number of features, not on making the core features work.  On the complete opposite side of the spectrum look at the iPhone.  A phone that has really high customer satisfaction year after year.  The iPhone doesn’t win all feature comparisons with other phones, yet the iPhone continues to sell really well.     The iPhone sells well not because it has more features than any other cell phone, but because the features they did build they built well.

5. Beware of the “home grown” LMS.

Too many support cases and forum posts begin with “Our in house developed LMS…”.  As I mentioned above, the SCORM standard leaves a lot open for interpretation.  It is naive to think that because there is a documented standard creating a SCORM based LMS will be easy.  There are many companies that (like Rustici Software) have made an entire business out of doing SCORM implementation, and if building an LMS and doing the SCORM implementation was easy, they wouldn’t be in business.

6. Hosted or Licensed?

Do you need to license the LMS and host it on your own servers, or is it alright for it to be hosted by the LMS company?

There are benefits and drawbacks to both approaches.

Benefits of licensed

• You own the software, and likely won’t have to pay re-occurring fees for anything but maintenance (if you choose to get it).
• You control the security - if security is a big issue for you, licensing the software might be better than a hosted solution because you can control access to the LMS (through a corporate firewall, etc).

Benefits of hosted

• Usually less expensive - If you outright buy the software you will likely be faced with a bigger upfront cost.
• Don’t have to manage the servers  - We for example host Articulate Online and Screenr at Rackspace.  Why do we do this instead of owning our own servers?  Well, Rackspace is the premier hosting provider in the world.  They are better at setting up servers, better at dealing with security, better at managing load balancing, fire walling, etc. than we are.  We handle the software part, which is a core competency of ours, and let Rackspace host the servers, which is a core competency of theirs.
• Updates are usually included for free – If you own your own server you may or may not be entitled to free upgrades.  We have many customers that are stuck using AICC because they licensed their server years ago, and no longer get free updates.  If you are on a hosted solution you most likely are always on the most up to date version of the software.

8. Is Open Source a possible solution?

I am not advocating this solution as the most painless, easiest option.  There are going to be a lot of implementation costs, time cost, hardware cost associated with an Open Source solution to your LMS needs, but Open Source should definitetly be considered.  The benefit of Open Source is the software is free.  The downside is that most Open Source Learning Managment Systems that I have used can be buggy, and hard to deploy.  I have only personally used Moodle and Dokeos (have used Moodle for about 4 years), and they have gotten better over time, but they still have some issues.

In addition to Moodle and Dokeos there are at least 10 other open source LMS options.

On another note, just because it isn’t open source, doesn’t mean they will have good support.  I will refrain by calling out any vendors by name.

Also there are some vendors that do open source hosting and support for the fraction of the cost of a traditional licensed LMS vendor.  Some names in this field are:

• Moodlerooms
• Kineo

9. What kind of reports are available?

Rustici Software (SCORM.COM) points out in SCORM for RFPs for LMSs that:

SCORM can provide a wealth of data about what a learner does in a course, but it’s up to the LMS to keep the data and use it. You’ll want to know what kind of reports there are, how data is tracked and what the learner sees at a minimum.

Really good point here.  Some Learning Management Systems do a really good job with capturing all the data, but don’t do the greatest job on reporting the data.  For example, Moodle does a good job at capturing all the SCORM data, and will do a great job of reporting status and score, but gets lazy with the reporting of the interaction data (how people answered each question).  If you want to see this in action, check out this post I did on uploading and tracking in Moodle.

A big part of the success of Articulate Online has been the richness of our reports (see this Articulate Online report presentation for more info).  With SCORM 2004 reporting, the LMS should be able to generate reports that are just as rich as what Articulate Online offers, but few LMS actually provide such elegant reports.

A good idea is to list out what type of information you will need to gather in your LMS, and then ask the LMS sales representative how to get that data.

For example, you might need to:

• Know who has taken a quiz
• How that person scored
• What questions are people missing consistently
• What is your most popular contet

Make sure your LMS is capable of generating the data you need.

10. How well do they work with content vendors?

During our beta cycles I occasionally reach out to LMS vendors to ask if I can test our content in their LMS just to make sure that everything works ok.  Most LMS vendors a really cool about this, and allow us to test in a trial or sandbox account.  But occasionally some of the bigger vendors ask for a substantial amount of money ($2000-$5000) just for us to test content in their LMS.  Seriously, and then they claim the charge is “nominal”.  Anyway, ask your LMS vendor if they partner with any content vendors, or if the authoring tool you use has been tested in their environment.

More information

SCORM.COM has a really good post on What to ask about SCORM in an RFP that has 7 questions anyone should ask of the LMS when getting a request for proposal and (at least) 27 questions to ask about your LMS vendor about SCORM.  Highly recommend checking it out.

Do you have any suggestions that you think should be added to the list?  Do you hate your LMS vendor?  Tell me about it in the comments.

Note: Someone pointed out this list only contains 9 items (I skipped 7).  Going to keep it titled a list of 10 though, because 10 reads better than 9  Sorry to short change you by 1.

We (at Articulate) are working on developing some Screenr screencasts that show you how you can upload and track Articulate content in various Learning Management Systems.  I got a head start on the project by doing a screencast on tracking Quizmaker in Moodle a couple of weeks back.  In this screencast I am going to show you how you can upload and track Quizmaker ‘09 content in Dokeos.

Dokeos is a Free Open Source LMS like Moodle.  Doesn’t contain nearly the feature set of Moodle, but it is a little simpler to use.  Enjoy the demo!

I have posted from time to time on the Open Source LMS Moodle, which is a pretty nice, free, and Open Source Learning Management System.  There is another Open Source Learning Management System on the market that is worth looking at as well, it is called Dokeos.  Seems a little bit simpler than Moodle, and a little more Web 2.0ish.

Some cool features of Dokeos:

• Offer video conferencing (I have yet to check this out)
• Fairly straight forward content upload process
• Cool Flash tutorials which makes figuring out how to do stuff very easy

I got dokeos installed on my server pretty easily, it only took about 30 minutes.

Took me a bit to figure out how I could upload my own content to dokeos, but finally stumbled across a tutorial on how to import SCORM content into dokeos.

Anyway, let me know if any of you have tried it, and what you think of it if you have.

Have you ever run into a scenario where you wanted to test your content in an actual LMS environment, but didn’t have an LMS to test in?  Or have you ever just wanted to play around in an LMS and see what is available?  Well I got some good news.  I use a hosting company called Dreamhost to host this website and a whole slew of other websites I have, and one of the really cool features they have is the ability to do a one-click install of Moodle on any of your websites hosted with them.

If you haven’t heard, Moodle is an Open Source LMS that is free.

Some of the cool features of Moodle:

• It’s Free!
• It supports SCORM 1.2
• It also supports AICC
• They plan on supporting SCORM 2004
• It supports and works with Articulate Presenter, Engage, and Quizmaker

The problem with Moodle is that it is a tad difficult to configure yourself, and upgrading versions is often a pain in the ass.  Well, Dreamhost has made installing and upgrading moodle versions really easy.

Dreamhost is a web hosting provider that will host your website, and offers a bunch of custom one-click installable plugins.

Some of the cool features of Dreamhost:

• Only $5.95 per month to host your domain
• Offers free one-click install for Moodle
• Allows you to host unlimited domains
• Offers free one-click install for wordpress so you can start blogging
• Offer a slew of other one-click installable software

It is pretty cool, and I highly recommend checking it out, as it is really cheap.  But I wouldn’t recommend using this as an enterprise level solution if you need an LMS.  The hosting provider charges only 5.95$ per month, so that kinda tells you that you shouldn’t expect a really high level of quality of service.  They also offer no additional support for the 3rd party plug-ins such as Moodle, so if something is not working in Moodle, they likely will not be able to help you.

We used Moodle on Dreamhost very extensively for testing the Articulate Studio ‘09 products, so our products are guaranteed to work with Moodle.

You can sign up for a dreamhost account here:

http://www.dreamhost.com/r.cgi?182057

I’ll be interested to hear if anyone else checks it out, so please comment if you sign up and try it out.

-Dave

The SCORM 2004 3rd edition was just released within the last year (I think) and the SCORM technical writing group has announced plans for SCORM 2004 4th edition.

Mike Rustici of scorm.com blogs about it here. He has some interesting things to say about the update, and how he feels it is necessary for progress. Also contains some interesting comments on a possible upcoming SCORM 2.0, which is in his words, “a radical update and restructuring to take advantage of how technology has evolved over since the original inception of SCORM and its underlying specifications nearly a decade ago.”

Mike Rustici is THE guy to talk to when it comes to SCORM/LMS integration, so his post makes for an interesting read.

Do you use a SCORM 1.2 based LMS with Articulate Quizmaker content and wonder why your LMS is truncating the responses to essay questions to only 255 characters?

The issue is actually that the SCORM 1.2 standard only allows questions repsonses to be up to 255 characters.

If you have worked with Learning Management Systems and content authoring tools such as Articulate Presenter, Quizmaker, and Captivate you might have run into issues with your content integrating with the Learning Management System . Often times finger pointing will happen between the Learning Management System and the authoring tool. The LMS company will say it is a problem with the content, then the authoring tool vendor will tell you it is a problem with the LMS, but very seldom do people ever point the finger at the actual standard as the culprit.

The more you find out about the SCORM 1.2 standard the more you begin to realize how inflexible it really is.

Note: The finger pointing I am reffering to is mainly from my experience at Adobe/Macromedia. Finger pointing doesn’t happen at Articulate

Every so often I see a forum post, or get an email where someone asks if Articulate supports multiple SCOs or can you repackage your Articulate content with other course content into a single SCORM-Compliant package. So, what is a SCO you ask?

Well, I lifted this from the ADL’s website, that explains what it is.

The Content Model defines a powerful model for breaking content into arbitrarily sized units of reuse. These units are called Sharable Content Objects (SCOs) and Assets. An Asset is simply an “electronic representation of media, text, images, sound, Web pages, assessment objects or other pieces of data.” Examples of Assets include images, sound clips, Flash movies, etc. A SCO is a collection of one or more assets that represents a logical unit of learning.

For more info on the SCORM communication process check out this site, it is a great resource for anyone looking for an Overview of SCORM communication.

So now that you know what a SCO is, you are probably asking, “well, why would I want to repackage to support multiple SCOs?”

The reason you might want to repackage your content is that maybe you want one single package that has both a Articulate Presenter presentation, and a Articulate Quizmaker quiz. Used independently, you could see that each one could be considered a different logical unit of learning. This can be useful becuase it allows you to reuse existing content, then just repackage several individual units of learning, and have it all in one nice little package.

Make sense? Probably not, but if you got here, and you are reading this, you probably already know why you would want to do this. So your next question is ok, I know what it is, and I know why I want it, now how do I repackage my individual SCOs into a single package. The answer is to use the Manifest Maker to repackage your content. So how do you use Manifest Maker? Well, I have included directions below from a recent forum posting I did.

In order to create multiple SCOs you will have to create multiple assests (for example a Articulate Presentation, and a Quizmaker Quiz) and create a new package using “Manifest Maker” to point to the launch files of each project.

Note: These directions will only work for SCORM 1.2 content and require the use of Macromedia’s Dreamweaver.

Install “Manifest Maker”

First you will need to download and install the “Manifest Maker” for Dreamweaver:

1. Go to the Dreamweaver Exchange
2. Under ‘Categories’ select ‘Learning’
3. Find ‘Manifest Maker’
4. Follow the prompts for installing the ‘Manifest Maker’

Note: For more information on the ‘Manifest Maker’ see the following website:
http://www.e-learningconsulting.com/products/manifest-maker.html

Publish Content

Now that you have installed “Manifest Maker” you must create your individual SCOs. For example, you can publish a Quizmaker Quiz, and a Articulate Presenter Presentation as each of your individual SCOs.

1. Create an output folder for both published projects
2. Create two subfolders in the output folder (one for each project)
3. Publish the projects to the respective directories with SCORM output enabled

Repackage

Now that you have created your individual SCO’s you will have to repackage them using the “Manifest Maker” inside of Dreamweaver.

1. Add your published Quizmaker projects to your Dreamweaver site
2. Open up Dreamweaver
3. Select ‘Commands’
4. Select ‘Manifest Maker 1.2…’
5. Set the appropriate fields on the ‘Metadata’ tab
6. Select ‘SCOs’
7. Select each of the published HTML files and click ‘Add SCO’
8. Click OKThis will generate the necessary files for you to repackage your content so you will have a package with multiple SCOs.

This will generate the necessary files for you to repackage your content so you will have a package with multiple SCOs.

Final Thoughts
I know this is a pretty advanced topic, and it might be very confusing, if you do not understand this topic, and you want more info, please don’t hesitate to contact me via the comment links below.

Also, Articulate Presenter and Quizmaker can work in tandem without having to repackage the content, the only reason why you should use the Manifest Maker is if you want to keep your individual assets separated, but be able to repackage them and support multiple SCOs.

Finally, if you think this post looks eerily similar to technote on another website…well, it is, and I wrote that too

Have a great Memorial Day!

It seems like lately a lot of people have been asking if Articulate Presenter and Articulate Quizmaker work with Moodle. For those of you that don’t know Moodle is a course management system (CMS) – a free, Open Source software package designed using sound pedagogical principles, to help educators create effective online learning communities (that definition came from moodle.org, I have no idea what pedagogical means). For more information on Moodle please visit http://www.moodle.org.

The good news is that Moodle has a SCORM module which allows you to upload any standard SCORM package, which includes Articulate Presenter and Quizmaker content, to include in your course. Both Articulate Presenter and Articulate Quizmaker are certified as SCORM compliant by an independent ADL certification expert which means without having to do anything special Articulate Presenter and Quizmaker will work and report results to Moodle.

The reason this question has come up so often is that Moodle, like many Learning Management Systems, is very complicated and difficult to learn. In the brief video below I have included a demonstration on how to integrate Articulate Presenter content (with an embedded Quizmaker quiz) into Moodle.

As you can see in the demonstration Articulate Presenter and Articulate Quizmaker can work with and report results to Moodle.

Note: Moodle can be a very complicated system to install, like many other Open Source applications, if you are not an advanced computer user. I would recommend getting the help of an advanced user before attempting to install and use Moodle.

For more information on this subject please visit the following sites:
Articulate Presenter SCORM certification information.
Articulate
Moodle
Moodle and SCORM