
Eunsu Noh
#Algorithm #Personalized Learning #AI
After finishing a course, a question like this often remains.
“So, what should I take next?”
There are hundreds of courses on educational platforms, but finding the one you need right now is not easy. A similar title doesn't necessarily mean it will help your learning, and popular courses aren't always the appropriate next step.
The AI tutor of edutap, built by Coxwave's AgentX team, answers learners' questions and helps check their understanding through content like quizzes. On top of this, we added a feature that helps find the next course together, so when a learner asks "Recommend the next course" in the chat, they can receive recommendations directly within the flow of conversation.
There were three core principles behind the design of our course recommendations.
First, gathering candidates based on the learner's history of initiating conversations with the AI tutor in each course, along with course categories. Second, selecting courses and explaining the reasons for recommendations using an LLM. Third, ensuring that course discovery can continue even when certain steps are unavailable. In this post, we introduce the problems we considered and the choices we made at each step.
Course Recommendations Starting from Chat
When a learner asks, “What would be good to take after this course?”, the AI tutor identifies the intent of the question and triggers the recommendation feature. Learners can explore their next course right where they asked, without needing to navigate to a separate menu.

The recommendation response includes three elements:
Introductory sentence: Explains the context in which the courses are being recommended.
Recommendation cards: Display basic information such as the course title and duration, allowing learners to browse courses of interest.
Follow-up buttons: Enable continued exploration with ‘View more related courses’ and ‘Explore other fields’.
This response is generated in the following order: Gathering candidates → Selecting and generating rationales with an LLM → Linking course information from the database.
Gathering Candidates
① Finding Relationships from Learner Behavior

The first set of course recommendation candidates is identified using Collaborative Filtering. Collaborative filtering is a technique that finds common patterns in the behaviors or preferences of multiple users to recommend items they might be interested in. We identified relationships between courses based on which courses learners started conversations with the AI tutor in.
We chose EASE as our collaborative filtering model. EASE learns relationships between courses as a single weight matrix. It calculates weights through matrix operations without iterative gradient descent training, and when making recommendations, it computes scores for each course by multiplying this matrix with the user's interaction history. The simplicity of its model structure and inference method was the reason for our choice.
As an interest signal, we used whether a learner initiated a conversation with the AI tutor in that course. While it is not direct feedback asking about course preference, we utilized it as an indirect signal indicating that there was a point of interest or a need for help during learning.
The model is trained separately for each client. This is because each client has different course catalogs and learners. In addition, we applied normalization techniques to mitigate the tendency of recommendations skewing toward popular courses. The reference study is “Why is Normalization Necessary for Linear Recommenders?”.
The top 25 courses obtained this way are used as the first candidate group.
② Using Course Categories When History Is Insufficient
To leverage the history of conversation initiations with the AI tutor, that history must first accumulate. For first-time learners or clients who have just adopted the service, it is difficult to find sufficient candidates using collaborative filtering alone. This situation, where making appropriate recommendations is difficult due to a lack of data on users or courses, is known as a Cold Start.
To supplement this, we bring in courses from the same category as the current course as a second candidate group. We organize and query course lists by category in advance, and for clients without category information, the entire course catalog is used as the starting point for candidates.
When merging the two groups, collaborative filtering candidates are placed first. Then, up to 30 candidates are passed to the LLM. It is structured to prioritize candidates found from the conversation initiation history with the AI tutor, while allowing recommendations to continue with category-based candidates in situations where history is insufficient.
Selecting with an LLM and Explaining the Rationale

Once candidates are gathered, we must decide which courses to display among them.
While collaborative filtering scores reflect the history of conversation initiations with the AI tutor, they cannot directly explain what the learner currently wants. Therefore, we used an LLM in the next step. It looks at the current course information, candidate course information, and the learner's query entered in the chat together to choose the final recommendations.
The information passed to the LLM is as follows:
Current course: Title, category, description, titles of key sessions
Candidate courses: Title, category, instructor, duration, description
Learner's learning query: Recommendation request entered in chat, such as “What would be good to take after this course?”
The LLM selects 3 to 5 courses based on three criteria. It considers whether it connects naturally with the current content learned, whether it leads to the next step of learning, and whether it complements areas not covered in the current course.
For each selected course, it assigns a score from 0 to 10 and writes a one-sentence recommendation rationale.
“If you have mastered basic grammar, you can apply it to practical conversation.”
The goal is to explain what connection prompted the recommendation, rather than just showing a list of courses to the learner. This score is a recommendation score assigned by the LLM and is not a measured value of actual learning effectiveness.
Course Information Is Retrieved from the Database
We do not display the LLM's response directly on the screen as-is. We first verify whether the returned courses exist in the actual candidate list, and any courses not in the list are excluded.
We also do not let the LLM generate information like prices, thumbnails, and links. Based on the selected courses, these are queried from the database and attached to the cards.
We divided the responsibilities. The LLM selects courses and writes recommendation rationales within the given information. The course information to be displayed on the screen is fetched from the database. However, since verifying candidates alone does not guarantee the accuracy of the recommendation rationale, the quality of the explanation must be reviewed separately.
Continuing Course Discovery Even Without an LLM Response

If an LLM call is delayed or the response format is invalid, the steps where the LLM selects courses and writes recommendation rationales cannot be completed. In such cases, we display the course candidates secured in the previous step so that exploration can continue.
In the fallback list, candidates found through collaborative filtering appear first, followed by candidates found through categories. Since these are not results re-selected by the LLM to fit the learning query, recommendation rationales and the ‘Recommended’ badge are not displayed.
The purpose of this path is not to guarantee the same quality as an LLM recommendation, but to ensure that learners can still browse courses even if a temporary error occurs.
The fallback paths for each stage are as follows:
If there is no collaborative filtering model or result, category-based candidates are used.
If category information is missing, candidates are composed from the entire course list.
If the LLM result cannot be used, the gathered candidates are shown as an exploratory list.
In this way, operations are divided according to the data available at each stage, and the LLM recommendation results and fallback lists are also distinguished on the screen.
‘View More Related Courses’ and ‘Explore Other Fields’
Recommendations do not end with a single response. We provided two buttons so that learners can look at more options or explore different directions.
‘View more related courses’ finds new courses in the same recommendation direction. Already recommended courses are identified from the conversation history and excluded. Because recommendation results are saved in chat messages, it is structured to also utilize the conversation history as a recommendation history.
‘Explore other fields’ finds courses that do not overlap with the category of the current course. In this path, collaborative filtering is not used, and the LLM's selection criteria are also altered. It is configured to look at whether it can connect from a different perspective than the current study, and how knowledge from other fields can be helpful.
Both buttons can only be used on the most recent recommendation. This is because clicking a button on an older recommendation could cause a discrepancy between the recommendation the user is viewing and the conversation context the system refers to. We aimed to maintain context not only in recommendation results, but also in the actions that continue exploration.
Next Step
In this implementation, we gathered candidates using the conversation initiation history with the AI tutor, supplemented insufficient history with course categories, and added selection and explanations using an LLM. We also established fallback paths so that course exploration can continue even when individual components are unavailable.
Whether the recommendations are genuinely helpful needs further verification moving forward. The current interest signal is merely ‘whether a conversation was started with the AI tutor.’ From this alone, it is difficult to know what the learner understood and where they experienced difficulty.
Going forward, we plan to look into ways to reflect concepts missed in quizzes or question topics revealed in conversations into our recommendations. Finding candidates based on the semantic meaning of course content and establishing a framework to measure recommendation quality are also our next tasks.
We intend to refine edutap to go one step beyond answering learners' questions, helping them choose what to learn next.