Analyzed over 30,000 conversations for karriereveiledning.no

Over the course of a year, approximately 16,000 individuals contact karriereveiledning.no seeking guidance on education and employment. How can we automate the extraction of structured information from over 30,000 anonymized conversations?

This is one of the challenges that Computas’ consultants have been working on in collaboration with the Directorate for Higher Education and Competence (HK-dir), which operates the website karriereveiledning.no. On the website, anyone can speak with a career counselor and receive free guidance on important decisions regarding education and employment.

In partnership with Computas, HK-dir initiated a project to analyze over 30,000 conversations between seekers, referred to as users of the service, and career counselors. The aim was to analyze the chats to gain a better understanding of how career counselors engage with users needing to explore their interests.

Decision-making Requires Accurate Guidance

The seekers who contact karriereveiledning.no are typically young individuals facing decisions that will have significant impacts on their lives. Therefore, it has been important for HK-dir to conduct thorough analyses of the conversations in order to further develop the service:

“We develop content for the website based on the idea that chats in the career counseling service are the best source for identifying end-users’ needs,” says Julie Utler Gjengedal, web editor and technical manager of karriereveiledning.no, adding:

“Computas has collaborated with HK-dir through a long-term ICT agreement, and therefore we knew that they had good knowledge of both the technologies and our data. The collaboration has taught us a lot about how we can work with AI, machine learning, and statistics. In addition, we have been challenged on how to think about extracting information from large datasets.”

Large-scale Analysis of Conversations in Free-text Format

Computas’ consultants utilized data-driven methods, as well as various models, to classify and identify patterns in the conversations between career advisors and seekers. Generative AI, search, and classical machine learning-based methods were used to determine the most effective way to analyze the conversations.

A significant challenge in the analysis work was that the conversations between seekers and career advisors are written in free-text format. This means that complicating factors such as typos, abbreviations, and dialects often arise in the dialogue. While it was sometimes difficult to analyze the data, the work and mapping provided valuable insights into what needs to be in place to effectively analyze conversations in the future.

“Since there were no available datasets of conversations where a career counselor assists the user in exploring their interests, the task was divided into two parts. The first part focused on identifying what actually constitutes interest exploration conversations, and the second part focused on analyzing these conversations,” says Data Scientist at Computas, Dharshini Tharmarajan, adding:

“In the end, we identified several parameters that characterize interest exploration conversations. Some examples include the duration of the conversations and the number of questions asked by users of the service.”

Launching new Tools for Interest Assessment

Insights from the analysis work in the project will be used in the development of a tool for interest exploration for youth who are choosing high school. HK-dir will launch the first version of the tool in December this year, and the innovation will be introduced on both Karriereveiledning.no and Utdanning.no.

“We now have valuable data that we can use as we work on creating a new tool for interest exploration this year. The results of the analyses can also be used in the development of the service, including in curriculum development. Additionally, we have learned a lot about how specific we need to be to extract good data,” concludes Gjengedal.

In the time to come, HK-dir will continue developing tools to assist seekers in choosing education or finding a job.