Beyond Academic Credentials: Assessing the Utility of Competency Assessment Tools
Chiedza Pasipanodya, Program Manager, Knowledge Management, Global Talent Bridge, WES
Immigration continues to be key to Canada’s economic prosperity. New arrivals accounted for a remarkable 71 percent of population growth and 90 percent of labour force growth in recent years, according to the Conference Board of Canada. While the COVID-19 pandemic has led to a temporary spike in the unemployment rate, the country faces long-term employment challenges as well. Given Canada’s declining birth rate and rapidly ageing population, maintaining economic growth and filling open positions will require the skills and expertise of immigrants.
Despite the evident and growing need for immigrants and the existence of well-established credential evaluation services, employers don’t fully understand or recognise the skills and competencies that immigrants offer. A 2019 WES report, Who Is Succeeding in the Canadian Labour Market? Predictors of Career Success for Skilled Immigrants, found that only 39 percent of survey respondents worked in jobs that were largely similar in type and complexity to their jobs pre-immigration. This study was based on data collected from a sample of 6,402 people who applied for a WES Educational Credential Assessment (ECA) between 2013 and 2015, and who were subsequently admitted to Canada as permanent residents.
Reports highlight a number of barriers to commensurate employment, such as employers’ informal requirement that candidates have “Canadian work experience.” These obstacles suggest a disconnect between the skills immigrants possess and employers’ understanding and recognition of those skills. As introduced in the first article in this series, “Competency Assessment and the Future of Work,” competency assessment is a possible solution to the ongoing underutilization and underemployment of skilled immigrants.
In a related report, Beyond Academic Credentials—Toward Competency-Informed Hiring, World Education Services (WES) defined competencies as the ability to apply knowledge and skills with appropriate judgement in a defined work setting. Competencies can be acquired through formal, non-formal, and experiential learning. As more employers look beyond academic credentials towards competencies, the need for tools and processes that assess what workers know and what they can do in specific environments has increased.
Despite the growing importance of these tools and processes, there is no consistent set of principles or standards that employers can use to determine their utility. As new tools emerge from different geographies and business sectors, employers and career service professionals must have a set of criteria against which they can evaluate the tools’ ability to meet their unique needs.
This article introduces a set of 7 principles and 16 descriptive statements WES has developed to provide end users, such as employers and career service professionals, with that criteria. These principles and the accompanying descriptive statements emerged from a project that developed a global inventory of competency assessment tools and practices. It is our hope that career service professionals will be able to use these criteria to ascertain which competency assessment tools and practices are best suited to support job seekers and increase service provision capacity.
Principles and Descriptive Statements
While we conducted research and engaged stakeholders to complete the WES report Beyond Academic Credentials, it became apparent that the most effective competency assessment tools shared several characteristics. We realised that these characteristics would need to be refined and tested in order for us to develop a set of principles that employers, career service professionals, and immigrants could use to promote immigrant inclusion. While we sought to develop these principles with immigrants in mind, we hoped that what we developed would be equally applicable to non-immigrant job seekers, and that the principles would become more valuable as more end users adopt competency-based hiring practices.
To design a global inventory document for use in our research, analysis, and evaluation of currently used competency assessment tools and practices, WES collaborated with students at the Munk School of Global Affairs and Public Policy at the University of Toronto through the Public Good Initiative (PGI). The PGI is a student-led initiative that provides pro bono consulting services to local community organizations while allowing participating students to obtain work experience in a professional setting.
To ensure that relevant information was captured in a consistent form, we developed a standardized template for use during the inventory process. The scope of the information gathered enabled us to compare different competency assessment tools across geographies, occupational sectors, and other axes. Key information captured also included where the tool was developed, where and in which languages it was operational, how it was accessed, how much it cost, how widely it was used, and what key stakeholders were involved in its development and currently use the practice or tool.
An example of a completed form can be found here. It captures relevant information about the FAST (Facilitating Access to Skilled Talent) platform developed by the Immigrant Employment Council of BC (IEC-BC).
After completing an environmental scan of globally used competency assessments, we produced an initial list of 70 tools and practices. We grouped these tools by geographic region of origin, such as Asia, Europe, North America, and Oceania, and later carried out an in-depth review of all the practices. We then identified seven key principles common to each of the most effective competency-based assessment practices. To increase the principles’ functionality and ease of use, we further developed 16 unique descriptive statements to clarify the essence of the principles.
The 70 tools were then analyzed and compared to the principles and descriptive statements. After lesser-quality tools were removed, the inventory of tools was reduced to 40. The attributes we noted in lesser-quality tools during this winnowing process, such as a lack of peer-reviewed evidence for the predictive powers of the tools, or the delivery of analog assessments online without fully leveraging technological innovations in data capture and analysis, seemed to confirm that we had identified the right set of principles. It is also important to explore the ethical and appropriate uses of AI; this will be touched upon in the third article in this series. The global inventory of competency assessment tools and practices is a living document to which further applicable tools can be added upon assessment.
The template below lays out the key principles and descriptive statements. These principles and statements can be used to assess the utility of a competency assessment tool or practice. Some statements apply across principles and appear under more than one principle.
| Key Principle | Descriptive Statements |
| Practical | The practice facilitates connection between job seekers and employers. |
| The practice can easily be integrated into existing networks and platforms (LinkedIn, etc.). | |
| The practice can be easily integrated into existing enterprise management and/or human resources management systems. | |
| Adaptable | The practice is responsive to labour market supply and demand. |
| The practice is able to identify the skills and competencies needed for new and emerging occupations. | |
| The practice can be used in different regions and cultural contexts; it is globally applicable and regionally relevant. | |
| Collaboratively Developed | The practice is developed with input from key stakeholders such as employers, regulatory bodies, and academic institutions. |
| The practice incorporates ongoing monitoring and evaluation and is responsive to stakeholder feedback. | |
| Standardized | The practice references or relates to a developed taxonomy (e.g., ESCO, ONET, NOC). |
| The competencies and skills identified by the practice are consistent with those used by key stakeholders such as employers, regulatory bodies, and academic institutions. | |
| Recognized | The processes and evidence of an individual’s competencies are understood and accepted as valid and valued by key labour market actors including job seekers, employers, assessment/accreditation organizations, sector councils, regulatory bodies, etc. |
| The competencies and skills identified by the practice are consistent with those used by key stakeholders such as employers, regulatory bodies, and academic institutions. | |
| The practice has been tested and accepted by key stakeholders such as employers, regulatory bodies, and academic institutions. | |
| Accessible | The practice is easy to understand and use for individuals with varying levels of literacy, including digital literacy. |
| The practice does not preclude individuals from participation due to prohibitive cost or time requirements. | |
| Licensing and distribution models are efficient and encourage adoption. | |
| Open | The practice can easily be integrated into existing networks and platforms (LinkedIn, etc.). |
| The practice can easily be integrated into existing enterprise management and/or human resources management systems. | |
| The practice complies with all relevant privacy legislation and adheres to best practices in ethical use of technology and artificial intelligence. |
Exploring a Tool’s Effectiveness and Relevance Using Descriptive Statements
The descriptive statements WES developed are designed for practical application and allow career service professionals to determine which tool best meets their clients’ needs. As discussed in the first article in this series on the pilots WES conducted, technology-enabled assessment tools, when well-designed and developed, can be used to increase the capacity of service providers while simultaneously empowering job seekers.
Following a needs assessment and an agency-specific, process-mapping exercise, career service professionals can use the WES-developed principles and descriptive statements to narrow the larger list of tools or practices to those that would best serve their specific needs. At this stage, limitations such as licensing restrictions, cost, and language should be identified. Once the practitioner or the agency has identified the tools best suited to their needs, they can make case-by-case determinations as to which tool would best serve clients’ individual needs as identified during the intake process.
Assessing various tools against the descriptive statements allows in-agency job developers to identify tools that are responsive to labour market supply and demand, and which allow immigrants to identify their skills and competencies, facilitating the process of connecting them to relevant jobs. Alternatively, general career counsellors may find more value in tools that better identify and highlight skills and competencies from different geographies and labour markets to help newcomers translate and express their education and experience in the Canadian context.
The criteria allow end users to identify a wide variety of tools and practices. While the capabilities of these tools are varied, in general, they can be used to do the following:
- Identify and assess a job seeker’s relevant competencies, and highlight sectors in the local labour market where these competencies are in demand. This capability facilitates immigration and career planning by allowing immigrants to identify their skills and experiences and understand the various ways those skills can be applied in different regional labour markets.
- Identify competencies that are both globally applicable and regionally relevant. A tool that captures labour market data across multiple countries would empower job seekers, allowing them to “translate” their job titles and experience from their home country into terminology and a skill taxonomy that would be recognized in Canada. Tools developed with global geographies in mind are also more likely to offer services in multiple languages, increasing accessibility and the empowerment of newcomers.
- Identify skills that need improvement and opportunities for training. Effective tools are responsive to labour market supply and demand and capable of identifying in-demand skills and sectors. They can also identify gaps in a candidate’s skill set and help create a plan to retrain or upskill so the candidate can remain competitive. Tech-enabled assessments can facilitate lifelong learning by integrating training programs as part of dynamic career pathways.
The right competency assessment practices provide practitioners with additional tools to ensure a holistic view of clients and their competencies that goes beyond education to draw on various aspects of their lives. Employed effectively, technology-enabled assessment tools can significantly augment the work that career counsellors do. Effective tools can also empower job seekers to direct and develop their knowledge, skills, and abilities towards greater career pathway exploration.
Conclusion
The number of competency assessment tools on the market will continue to increase as more employers and career service professionals seek to better understand the skills available and the tools’ value and potential application in the labour market. When a number of tools are being assessed, a set of standards must be available to measure each tool’s relevance and utility; the principles and descriptive statements set forth in this article are a step towards the development of consistent and widely accepted frameworks to assess competency.
There is value in harnessing these tools to improve skills recognition, to ensure that immigrants work in commensurate employment, and to reduce skills underutilization. A CIBC report found that the underemployment of immigrants costs the Canadian economy more than C$20 billion in forgone earnings. Assessment tools should be used to translate newcomers’ skills and experience, and to facilitate the equitable access of immigrants to the labour market.
The final article in this series explores how competency-based interventions can be used to bridge the disconnect between immigrants and employers. Given the demographic challenges Canada faces, it is imperative that we find ways for immigrants to contribute to the economy at a level commensurate with their skills. By providing a standardized, informed way for employers to acknowledge the education and experiences of immigrants in a way that is familiar and transparent to them, competency-based assessments can improve immigrant employment outcomes.
We acknowledge the support and contributions of the following students at the Public Good Initiative (PGI): Timothy Gopaul, Karishma Firdausi, Omar Ali, and Reem Sheikh-Khalil with support from Kristi Kodama.
The views and opinions expressed in this article are those of the author(s) and do not necessarily reflect the official policy or position of World Education Services (WES).