Abstract
This review investigated findability and its relationship to usability and usability evaluations. Definitions of usability, findability, usability evaluations, and other sub-components of these terms including navigability are discussed, and the importance of research studies using similar terminologies is considered. A series of six usability evaluation studies is presented, and the terminologies presented in each are described. Goals, findings, and conclusions of each study are considered, and applications from the conclusions are drawn. This review concludes that clear definitions of these topics are crucial to successful implementation of various study methodologies and presentation of results. The review further concluded that, based on the available research, findability is a critical factor of usability that, when effectively present, leads to greater usability and overall user satisfaction of a product or website.
Keywords: Usability evaluations, findability, accessibility, usability
Introduction
Usability Evaluations
Usability evaluations are systematic reviews to evaluate usability. Trivedi & Khanum (2012) define usability as “the ability of a system to carry out specific tasks by specific users in a specific context” (p. 1). In this way, usability is concerned with how users interact and utilize a system to meet the objective of the system, given a specific circumstance or time. Usability evaluations, then, are concerned with evaluating how well usability is achieved given a particular system.
Research studies and industry studies often utilize usability evaluations in order to explore usability of a system or product. Usability evaluations are often considered quick and relatively cheap means of evaluating usability (Trivedi & Khanum, 2012). Nonetheless, usability evaluations can quickly become complicated considering the multiplicity of available evaluation techniques, models, and usability factors that are involved. Additionally, because of this highly complex nature, it is difficult for inter-study comparisons to be completed, leading to confusion in the literature when comparisons are attempted (Hartson et al., 2001). Researchers have argued that a standardized method or model should be developed for use in usability evaluations, but studies remain diverse in their selection of methodology, models, techniques, and even factor definitions.
There are many models and techniques that can be used to frame usability evaluations. However, some models are more common or usable than others. A common model used in modern research is the Nielson Model, named after a key figure in the development of usability studies (Muqtadiroh et al., 2017). The Nielson Model evaluates a specific set of usability factors, conceptualized by results of rigorous research. These conceptualized usability factors in the Nielson Model include findability, learnability, efficiency, memorability, few errors, and user satisfaction (Muqtadiroh et al., 2017). Certain usability evaluation techniques include heuristic evaluation, cognitive walkthroughs, and action analyses (Trivedi & Khanum, 2012). Other models and techniques include other usability factors. Importantly, there are many factors that can be evaluated, and each of these factors often have multiple measures (Banker, 2020). Usability studies thrive on the interplay among these factors and the techniques which are used to obtain the data about the factors. Conceptualizing and defining the terms, models, and used techniques are important for clear communication, understanding, and successful usability evaluations.
Findability
Findability, as mentioned above, is a usability factor that is specifically examined in usability evaluations. Findability represents how findable something is (Banker, 2020). In the context of usability evaluations, findability is often concerned with how findable particular data are, but findability could be concerned with finding any aspect of a system or product. Findability has been measured in a variety of ways overall, but a prominent set of measures of findability are navigability and retrievability (Wilkie & Azzopardi, 2013; Banker, 2020). Wilkie & Azzopardi (2013) describe navigability as how easily it is to navigate a site’s link structure while looking for a specific page, and they describe retrievability as how easily a user can retrieve a page using a search engine. Further definitions of retrievability emphasize that retrievability is focused on what, how, and whether a user is likely to retrieve something or not instead of simply relevance (Azzopardi, 2015). While these particular definitions tend to lean in the direction of website or webpage findability, the concepts could be used in other domains as well.
Purpose
The purpose of this review is to analyze usability evaluation studies and explore specifically the findability usability factor in the literature. This review will address usability evaluations and their effectiveness and the role of findability in these studies. For the purpose of this review, usability evaluations seek not only to understand the current state of usability of a particular system or product but also to improve the usability of that system of product for a better overall user experience (Trivedi & Khanum, 2012).
Review
Navarrete & Lujan-Mora (2015)
A usability evaluation was conducted to examine the usability of e-Education collaborative environments which utilize Open Education Resources (OER) (Navarrete & Lujan-Mora, 2015). Researchers describe the importance of OER in the changing educational landscape of the past decade but emphasize the often-lacking accessibility due to issues of findability. Because of these findability issues, OER can actually become barriers, specifically for users with disabilities, for the accessibility of e-Educational resources which aim to increase the accessibility of these resources to greater numbers of users. This team defines findability as how easy it is to find content or functionality that the user expects on a particular site, which would be specifically OER sites in this case. In order to examine findability, Navarrete & Lujan-Mora (2015) explored three measures of findability, namely web accessibility, web usability, and information architecture. Evaluation of each measure differed, but researchers defined a low score in any of the measures as detrimental to the findability of the examined OER. Researchers conducted the usability evaluation focusing on findability of educational resources from the perspective of users with disabilities utilizing a set of 247 guidelines extracted from international standards (i.e., ISO). Navarrete & Lujan-Mora (2015) found that each site they examined, as well as many of the educational resources available on these sites, presented findability concerns for users with disabilities, suggesting that each site should be re-evaluated in order to increase the intended accessibility. A proposed takeaway from this research is to increase awareness of usability factors, specifically findability, in these OER sites during development, evaluation, and modification (Navarrete & Lujan-Mora, 2015). This study highlights how findability can directly impact accessibility of resources for different groups.
Shieh (2012)
A usability evaluation study to evaluate the usability and findability of electronic library resources was conducted (Shieh, 2012). In a similar goal to the aforementioned study, this study aimed to evaluate the usability of these electronic library resources to improve accessibility through website usability and specifically to internal website resource findability. The study author defines effective findability as the ability to easily find a user’s required information both quickly and intuitively (Shieh, 2012). The study methodology specifically entailed an interesting approach to reconstruct the accessed library websites and resources using user logs and sub-sessions. In this way, the log information of the actual sessions indicating how a user was using a site at the time were used to reconstruct the sites, which emphasizes the actual use-case of the site overall. The reconstructed library sites were used to evaluate overall findability using a heuristic method. Using this log recreation model followed by heuristic evaluation, Shieh (2012) found increased findability on the reconstructed web pages. This novel model could be used in future research for research or business teams hoping to increase findability on a particular web page.
Auinger et al. (2012)
Auinger et al. (2012) conducted a theory-based usability evaluation to examine how findability and usability interact with search engine optimization (SEO) for business success. Researchers in this particular study fail to adequately define findability, which was noted earlier in this review to be a concern with these types of studies. However, the researchers emphasize the functional importance of findability as the most critical factor for effective usability and accessibility. Auinger et al. (2012) define usability per the ISO Standard 9241. Researchers utilized a scenario-based usability evaluation consisting of four different scenarios to examine the effects of the different kinds of SEO-optimization techniques defined in the study. Each scenario included an SEO-optimized version of the site to be studied as well as the current, live, publicly accessible version of the site. Users participated in the scenarios and answered questions concerning the use of the sites in their respective trials. Researchers found a positive effect of SEO-methods on site usability while finding negative effects on usability when SEO-problems were present (Auinger et al., 2012). Researchers conclude that websites with SEO-methods applied are more usable and lead to higher satisfaction due to greater overall content and information findability. Overall, this study should inform other investigators and business teams to consider SEO-methods to increase website findability and overall usability.
Cho et al. (2019)
A study designed to examine usability and accuracy of a newly developed pill identifier application that can be used on smartphones was conducted (Cho et al., 2019). This study differs from the previous studies evaluated because it does not directly consider findability as a crucial component of the product. Rather, findability is simply one of six measured facets of usability from the honeycomb model. An experimental design to compare the new image-processing based pill identifier application to the existing conventional pill identifier application and usability evaluation yielded data suggesting increased findability of the image-processing based pill identifier application (Cho et al., 2019). Researchers conclude that, overall, accuracy was not statistically different between the new application and the existing application, but usability of the new application was nonetheless good when used by medical personnel. The researchers suggest that usability for this application needs to be extended to other groups in the future for greater usability between groups. Nonetheless, findability was greater in the new application, and these results may impact the suggested choice of application for medical professionals in the future (Cho et al., 2019). The study results differ from the previous studies as well in that it is the only study evaluated thus far where increased findability did not correlate with overall increased usability. This particular study can be used as support for selecting the new application for medical professionals in settings requiring use of a pill identifier application.
Simunich et al. (2015)
Another study was conducted to examine findability and its importance in online student perceptions of satisfaction in online courses (Simunich et al., 2015). The purpose of this exploration was to determine if findability should be more greatly emphasized in online course design. Simunich et al. (2015) quote Morville (2005) for their definition of findability, which is the following: “the degree to which a particular object is easy to discover or locate, [as well as] the degree to which a system or environment supports navigation and retrieval” (p. 174). Researchers emphasize the foundational importance of findability in usability and usability evaluations, and they consider findability as crucial to online educational environments. Simunich et al. (2015) chose two online courses for control groups and compared them to modified versions of both courses. The modified courses consisted of the same content with altered design specifically tailored to go against usability standards. The researchers intentionally violated the usability standards for the experimental groups. Participants were randomly assigned to either one of the control groups or one of the experimental groups. In addition to one of these groups, users were assigned to either an eye-tracking group or a focus group for data gathering. Researchers found that findability predicted self-efficacy and motivation in online courses, where greater findability led to overall greater satisfaction in the online courses (Simunich et al., 2015). Researchers conclude that this research may be useful in defining minimum findability requirements for online courses in the future in order to lead to greater usability and overall student satisfaction, self-efficacy, and motivation in online courses.
Samuel et al. (2012)
A final study examined the usability of health information websites and findability of information on those sites (Samuel et al., 2012). Researchers in this study define findability as “the ease of locating information on a website” (p. 709). Researchers were specifically interested in the aspect of findability concerning content on the health information sites. Researchers identified a few health information sites to evaluate user interaction. Methodology included usability evaluation and online surveys with tree testing. Assets on the health information sites, specifically the search box, navigation menu, and home page were evaluated for usability and information findability using those assets. Samuel et al. (2012) found that the search box is the primary way that users find information on these health information sites. The navigation menus and links are seldom used. Because of this, Samuel et al. (2012) conclude that the search box should be the most prominent asset visible on the health information sites, especially on the home page which represents the first interaction a user will likely have with a particular site. Researchers suggest a variety of ways to improve navigability (discussed briefly in this review’s introduction) including adding faceted search to the search box. These recommendations are thought to improve findability, but more research will be needed in order to support these assertions. This study could be used to both support researchers and business professionals in their development of usable sites involved with information dissemination. Findability and emphasis on usable search tools should be highlighted in these cases.
Conclusion
This review evaluated the concept of usability evaluations and the foundational component of findability. Findability is a crucial usability factor that, when present and emphasized in effective ways, leads to more usable products and websites. The studies examined provide insight into a high-level understanding of how these usability evaluations are considered and approached. They also provide insight into the relationship of findability and usability, where greater findability tends to lead to greater usability and overall user satisfaction. The importance of considering definitions and conceptualizations of the various terms involved is evident when reviewing the literature. While many studies use similar definitions of findability, different definitions can lead to slightly different understandings of the importance of findability in these usability evaluations, which may lead to completely different approaches, biases, and considerations when designing studies.
References
Auinger, A., Brandtner, P., Großdehner, P., & Holzinger, A. (2012). Search engine optimization meets e-business-A theory-based evaluation: Findability and usability as key success factors. In ICE-B 2012: International Conference on e-Business and Telecommunications (pp. 237-250).
Azzopardi, L. (2015, March). A Tutorial on Measuring Document Retrievability. In European Conference on Information Retrieval (pp. 813-816). Springer, Cham.
Banker, D. A. (2020). Findability, Accessibility, and Usability of Data Portals in Education. Journal of Education & Social Policy, 7(1), 8-15.
Cho, S. K., Kim, B., Park, E., Kim, J., Ryu, H., & Sung, Y. K. (2019). Usability Evaluation of an Image-based Pill Identification Application. Journal of Rheumatic Diseases, 26(2), 111-117.
Findlater, L., & McGrenere, J. (2007, September). Evaluating reduced-functionality interfaces according to feature findability and awareness. In IFIP Conference on Human-Computer Interaction (pp. 592-605). Springer, Berlin, Heidelberg.
Hartson, H. R., Andre, T. S., & Williges, R. C. (2001). Criteria for evaluating usability evaluation methods. International journal of human-computer interaction, 13(4), 373-410.
Matera, M., Rizzo, F., & Carughi, G. T. (2006). Web usability: Principles and evaluation methods. In Web engineering (pp. 143-180). Springer, Berlin, Heidelberg.
Morville, P. (2005). Ambient findability. Sebastopol, CA: OReilly Media.
Muqtadiroh, F. A., Astuti, H. M., Darmaningrat, E. W. T., & Aprilian, F. R. (2017). Usability evaluation to enhance software quality of cultural conservation system based on nielsen model (wikibudaya). Procedia Computer Science, 124, 513-521.
Navarrete, R., & Lujan-Mora, S. (2015, April). Evaluating findability of open educational resources from the perspective of users with disabilities: a preliminary approach. In 2015 Second International Conference on eDemocracy & eGovernment (ICEDEG) (pp. 112-119). IEEE.
Redish, J., Bias, R. G., Bailey, R., Molich, R., Dumas, J., & Spool, J. M. (2002, April). Usability in practice: formative usability evaluations-evolution and revolution. In CHI'02 extended abstracts on Human factors in computing systems (pp. 885-890).
Samuel, H. W., Zaïane, O. R., & Zaïane, J. R. (2012, January). Findability in health information websites. In Proceedings of 2012 IEEE-EMBS International Conference on Biomedical and Health Informatics (pp. 709-712). IEEE.
Shieh, J. C. (2012). From website log to findability. The Electronic Library.
Simunich, B., Robins, D. B., & Kelly, V. (2015). The impact of findability on student motivation, self-efficacy, and perceptions of online course quality. American Journal of Distance Education, 29(3), 174-185.
Trivedi, M. C., & Khanum, M. A. (2012). Role of context in usability evaluations: A review. arXiv preprint arXiv:1204.2138.Wilkie, C., & Azzopardi, L. (2013, March). An initial investigation on the relationship between usage and findability. In European Conference on Information Retrieval (pp. 808-811). Springer, Berlin, Heidelberg.