The LMS Buyer’s Guide: How to Test Drive a Platform with Confidence

Selecting a learning management system (LMS) for your organisation is one of the biggest decisions any learning professional will make. That’s why we’ve created an LMS buyer’s guide that will support you in finding a platform that fits your needs and supports your learning goals. 

You’ve seen the sales decks. You’ve been wowed by lengthy feature lists. And you’ve watched the sales demos. But, these touchpoints are like a first date where everyone is on their best behavior. So, how do you avoid a costly and regrettable LMS marriage?

Well, you need a full-throttle, hands-on-the-wheel, test drive that puts the platform through its paces on your roads, with your learners, and your real-world content. In this article, we’ll show you how to take to the driver’s seat and evaluate the experience for admins, learners, and instructors alike.

Ready? Then let’s shift into high gear and start with an examination of why it’s so important to test drive your new learning system. 

Buying an LMS: Why a Test Drive is Crucial

Choosing the wrong learning management system (LMS) isn’t just a minor inconvenience. It’s a costly mistake. While the ATD found that 83% of organisations currently use an LMS, the landscape isn’t as rosy as that figure might suggest: 

• Fit: 50% of L&D pros claim their LMS is not a good fit for their workforce.
• Relevance: Only 22.9% believe their system solves the right problems.
• Satisfaction: This leads to 44% of organisations being unhappy with their platform.
• Action: Consequently, 1 in 4 are now planning to switch providers.

Yikes. LMS dissatisfaction has clearly reached a tipping point. Worse still, this is a high-stakes investment. Companies spend an average of $1,500 per employee annually on training, with a significant portion of that budget allocated to learning technology.

Unfortunately, a good LMS doesn’t come cheap. Setup fees can range from $4,000 to $40,000, and a monthly user licence can cost $2 to $5 per employee. 

Consider an organisation with 1,000 employees. Taking the median of these costs, they could spend $64,000 in the first year alone — and that doesn’t account for further integration and support costs. What’s more, this investment is often tied to a multi-year contract. 

If the system fails to deliver on its promises that money will be wasted. Worse still, you’ll be stuck with a system that frustrates both you and your learners. A proper test drive is your best insurance against this outcome. After all, you wouldn’t buy a car based solely on a spec sheet. 

You’d want to get behind the wheel and take it for a spin yourself. 

Navigating the Market

Okay, so we’re agreed that a test drive is essential. The next step in this LMS buyer’s guide is to figure out how to navigate this massive and complex market. There are over 800 different LMSs available, each with its own unique set of features and benefits. 

For example, here at Growth Engineering, we specialise in learner engagement, gamification, and social learning. However, other vendors may focus on skills, compliance, eCommerce, and so on. As you can imagine, this can easily lead to choice paralysis. 

Perhaps this explains why 23.4% of learning professionals spend more than six months evaluating the market before selecting their supplier. To ensure this time is well spent, you need to define your criteria upfront and stick to it (more on this shortly). 

Without a clear plan, you risk becoming part of the 33.3% of professionals who change their selection criteria during the purchasing process, wasting valuable time and resources. Don’t let this happen to you!

The reality is, most of us aren’t experts in this area. Indeed, only 45.4% of learning professionals consider themselves ‘very knowledgeable’ about LMS selection and implementation. Unless you’re a true specialist, this is likely to be a decision you’ll only make once or twice in your career.

The sheer number of options and the complexity of the market underscore the need for a strategic and disciplined approach. This preparation isn’t just about saving time, it’s about giving yourself the best possible chance to find the perfect fit. 

Phase 1: Preparation is Key

The success of your test drive hinges entirely on what you do before you even log in to a demo. Without a clear plan, you’re simply looking at a series of features without any context. As with any learning journey, it’s hard to get started if you don’t know where you’re going. 

This phase is about setting a solid foundation, ensuring your team has a clear understanding of what a ‘good fit’ actually means for your organisation. We’d break this down into the following four steps:

1. Define Your Needs

Before you speak to a single vendor, you should create a checklist of non-negotiable features, technical requirements, and business objectives. This will give you a north star to aim towards, preventing you from being swayed by flashy sales pitches. Here, you should consider:

• Learning Objectives: What are your primary goals? Are you looking to onboard new hires, upskill existing employees, or ensure compliance?
  
  
• User Experience (UX): How will learners access the platform? How important is learner engagement and should the platform be mobile-friendly?
  
  
• Admin & Reporting: Can you easily enroll users, manage groups and content, and pull the data you need to prove your programme’s ROI?
  
  
• Technical Requirements: Do you need to integrate the platform with your existing HRIS or Single Sign-On (SSO) system? What are your security and privacy requirements?
  
  
• Budget & Support: Finally, what is your total budget for the first three years, and what level of customer support will you require?

Use the answers to these questions to build your checklist. Keep it simple for now, and remember to categorise each item as a ‘must-have’ or a ‘nice-to-have’. 

Psst. Need some guidance on how to weigh these elements? According to Capterra, functionality is the top factor in any LMS purchase decision (39%), beating out reliability (20%), support (17%), and even price (12%). 

2. Assemble Your Team

Your new system will impact multiple areas of your business, so your test drive team should reflect that. You’ll want to involve key stakeholders from different departments to help build a complete picture. For instance:

• Learning & Development: Assign key members of your team to evaluate the platform’s ability to create, manage, and deliver content.
  
  
• IT & Technical: Find relevant resources to assess the platform’s security settings, integration capabilities, and technical infrastructure.
  
  
• HR & People Operations: Have stakeholders verify that onboarding is seamless, user management is intuitive, and compliance tracking is robust.
  
  
• End-Users: Crucially, bring in a few potential learners from a target group and gather their unfiltered feedback on the user experience. 

By getting diverse input early, you’ll uncover potential problems and gain buy-in from the people who will actually be using the system every day. This will set you up for a faster and more confident final procurement decision. 

3. Develop Use Cases

With your goals and team in place, your next step should be to develop 3-5 specific use cases. This hands-on approach will help to demonstrate the system’s real-world capabilities. You can ask the vendor to walk you through these scenarios or, better yet, try to perform them yourself. For instance:

• Onboarding new hires and assigning a learning pathway.
• Launching and tracking a compliance programme.
• Running a virtual instructor-led session.
• Authoring and selling eLearning content.
• Creating a social learning community.

Remember, there’s more than one way to skin a cat. Focus on the desired outcome, rather than being too prescriptive about a specific use case. This gives you more flexibility in how you use the platform.

4. Shortlist Your Systems

With so many different LMSs on the market, you can’t possibly test drive them all. So, before you move onto the next phase, you’ll need to narrow down your options to a manageable shortlist of 3-5 vendors that are a good fit for your needs. 

Now, it’s time to do your research. Make use of reputable review sites and consult with learning technology experts as necessary. You can also ask peers in your industry, other L&D professionals, or those in your wider network for recommendations. 

As you gather information, filter out any vendors who are unable to meet your ‘must-have’ criteria or those whose pricing is simply too high. By doing this, you’ll quickly reduce your list of hundreds of vendors down to a handful of strong candidates. 

Free Resources:

Your Interactive LMS
Purchase Checklist

Download Now!

Phase 2: The Hands-On Test Drive

Once you’ve completed your prep, it’s time to get your hands dirty. This is where you’ll move past the vendor’s polished demo and truly test the platform’s capabilities against your specific use cases. Typically, you’ll have two options here:

• Trial Accounts: A trial account gives you access to the vendor’s LMS as a regular user, often for a limited time (e.g., 14 or 30 days). While this is a great way to explore the user interface and basic features on your own, it may not include all the administrative tools or allow for full customisation.
  
  
• Sandbox Environment: A sandbox, on the other hand, is a dedicated, fully-functional replica of the vendor’s LMS, where you can experiment freely without affecting any live data. This is the ideal option if you’re looking to complete a comprehensive test drive, as you’ll have access to all platform features.

Remember, the goal is to experience the platform from all sides: the learner, the administrator, and the L&D professional. 

1. Assessing the User Experience

It goes without saying that your learners are the centrepiece of your training programme. As such, you should start by evaluating the user journey from their perspective. After all, it doesn’t matter if you have the best features in the world, if your employees don’t use the platform. With this in mind, here’s what you should be looking out for:

• Ease of Use: Is the platform clear, intuitive, and uncomplicated? Can a new user easily access their assigned courses and learning pathways?
  
  
• Content Access: How easy is it for your learners to explore the library and self-serve content, such as videos, documents, or quizzes?
  
  
• Accessibility: Is the platform accessible and responsive on various devices? Remember, many of your learners will access their training materials through a mobile device.
  
  
• Progress Tracking: Can your learners easily monitor their progress and completion status? Is it clear what they’re working towards?
  
  
• Learner Engagement: Is the platform fun and engaging to use? Are game mechanics and social features available to incentivise activity?

Need some additional support? This list of the 44 ‘Must-Have’ LMS features may be a good starting point. 

2. Assessing the Admin Experience

Next up, it’s time to switch to the administrator’s perspective. This is where you’ll test the features you need to manage your learning programmes efficiently. If this proves too complex, your training initiatives will struggle to get off the ground.

• User Management: As we’ve seen, your learners are the most important part of your training programme. So, can you easily add new users, create groups, and enroll them in courses?
  
  
• Content Management: How easy is it to upload, organise, and manage different types of content? Can you organise individual assets into learning pathways?
  
  
• Reporting & Analytics: Can you monitor progress and pull custom reports? Are the platform’s dashboards clear and comprehensive?

At this stage, you’ll also want to confirm that the system makes it simple to set up your use cases and achieve your learning objectives. For example, if you’re using the LMS for a compliance programme, check that it can handle features like due dates, automated reminders, and certificate generation. 

3. Integrations and Support

Finally, before you move on, you’ll want to check that both the platform and the vendor are capable of meeting your needs as a learning professional. This means you’ll need to check that they fit neatly into your existing ecosystem.

For instance, does the platform integrate seamlessly with your essential business tools, such as your HRIS (for user provisioning), Single Sign-On (for ease of access), and video conferencing platforms? Test these connections to ensure a smooth workflow. 

And don’t overlook the importance of vendor support. While you have access, why not test their responsiveness by submitting a support ticket? You’ll also want to assess their knowledge base, user forums, and see if they have a customer community where you can find answers and connect with other users.

A strong support system can make all the difference during implementation and beyond. 

Phase 3: Evaluation and Decision

After all the preparation and hands-on testing, it’s time to bring your findings together and make an informed decision. This phase in our LMS buyer’s guide is all about moving past opinions and focusing on a data-driven process to select the right system for you and your organisation. 

1. Score Your Findings

This is the fun part! Revisit the ‘must-have’ and ‘nice-to-have’ checklist you created way back in Phase 1. Now, add a new column where you can assign a score to each item. Remember to get feedback from the entire team (including end users) as you complete this process.

You’ll want to assign a quantitative score (for example, a 1-5 scale) alongside qualitative feedback on what your team liked and disliked. Be sure to capture any unexpected issues or pleasant surprises and try not to get caught up in features that won’t move the dial. For instance:

This will give you an overall platform score and transform your subjective feedback into tangible data. But, before you make a final decision, there’s a few other things you need to chew over.

2. Moving Beyond Features

While your test drive will primarily focus on features and functionality, your final decision should consider a broader picture. Remember, you’re likely to be working with your selected vendor for many years to come. Think of them as a long-term partner, not just a product provider. 

In other words, a thorough investigation is essential. What do other customers say about them? What is their long-term vision for the product? Are they actively developing and improving the platform? Consider every element that is important to you, and assign it a score, as in the example below:

You can also choose to assign specific weights to individual criteria. For instance, if pricing and support are the most critical factors for your organisation, you may want to apply a multiplier (e.g., 1.5x-3x) to those scores before calculating the final total. 

And there you have it. Armed with a clear scoring sheet and a full understanding of each vendor, you’re now ready to make a confident decision. What’s more, your test drive will have given you all the evidence you need to convince your stakeholders and get buy-in. Congratulations!

Final Words

Our LMS buyer’s guide is designed to help you move from the glitzy showroom of sales pitches to the open road of real, hands-on experiences. Ultimately, it’s about empowering you to make the best possible decision. 

You now know how to check under the hood for admin headaches, test the ride for learner comfort, and ensure it has all the features you need for the long journey ahead. Thankfully, that means no more guesswork and no more buyer’s remorse. 

Here’s to a future of awesome learning experiences on your platform of choice. Now, go find your perfect ride!

Thanks for reading. If you’ve enjoyed this content, please connect with me here or find more articles here. 

Need more support in finding your dream LMS? We can help. Get started with our interactive LMS Purchase Checklist, designed to simplify your search. Then, see for yourself by signing up for a demonstration of the world’s most engaging learning platform. Enjoy!

Free Resources:

Your Interactive LMS
Purchase Checklist

Download Now!