Software trials are an essential part of the library and information management role. We are constantly being bombarded with new products, services, solutions and databases, however, only the most promising will make it as far as a trial. Trials allow information professionals to test solutions in real-world scenarios and if done correctly, they are a key part of the implementation process.
However, not all software trials are created equal. Some are more successful than others, and this often comes down to the success criteria used. One question that you will be asked when talking with any vendor that you’re trialling is what are your success criteria. For example, Vable will ask you,
Define your success criteria - we ask that you compile a list of items that need to be tested and proven during your trial. What boxes need to be checked for this to be considered a successful evaluation and for you to move forward?
Success criteria are measurable standards that are used to determine whether a project or software trial has been successful. They are specific and measurable items that are used to evaluate the success of a project or software trial.
One definition says they are;
It is useful to look at some practical and real-life examples of good success criteria.
Good success criteria should be specific, measurable, achievable, relevant, and time-bound. They should be designed to test specific aspects of the product and should be aligned with the overall goals of the trial, as well as being tied into your overall information service strategy/action plan.
For example, if you’re testing a new current awareness aggregator like Vable, your success criteria might include things like:
As you can see there is a wide range of questions, metrics, and requirements to identify. However, in this list, there is a mix of good and bad success criteria. Which ones are clear and measurable? Which ones need more thought?
Bad success criteria are those that are vague, subjective, or difficult to measure. For example, a bad success criterion might be “we want to save time”, “the system should be easy to use” or “we want to send newsletters to clients”. These are subjective criteria that are difficult to measure and could mean different things to different people.
Ensure that all of your success criteria are specific and measurable, so everyone involved in the trial knows exactly what needs to be achieved. Don’t forget, this isn’t just something that you do at the start of the project, you need to be constantly measuring where you’re at.
It is important to document your success criteria so that everyone involved in the trial knows exactly what needs to be achieved. This documentation should be drafted at the start of the project and include a clear description of each success criterion, as well as how it will be measured.
The document should include:
The documentation should also include any relevant data or metrics that will be used to evaluate the success of each criterion. This can help to ensure that progress is tracked accurately and that any issues are identified early on in the development process. And remember, the success criteria should not be set in stone - agility and the ability to react quickly are essential for project management.
Having clear success criteria can identify issues that may arise during the trial which ultimately saves time for everyone. If the product does not meet the success criteria, this may be an indication that the product needs more work. This is a positive finding because these insights can help vendors with their development roadmap, ensuring a better future product for everyone.
Success criteria help you focus your attention on a particular piece of software, a product or whatever it is you are trialling. However, getting into the habit of measuring, testing, checking and constantly evaluating something also offers other benefits. As one project manager reminded me, there are many positive reasons for constantly measuring the effectiveness of a project:
Success stories, usage and other relevant metrics that demonstrate the implementation progress can be incorporated into roadshows and presentations to justify the investment of time and resources in ongoing technological investments and training programmes.
By having clear outcomes to report to team members, managers and budget holders, you will be in a stronger position to clarify what value you are adding to the organisation.
Having clear success criteria is essential for ensuring a better trial experience. They help to ensure that everyone involved in the trial - including your vendor - is working towards the same goals and that there is a clear understanding of what needs to be achieved. This can help to reduce confusion and misunderstandings during the trial and can lead to better results.