Mobility Repository

Webinar 5 – QUESTIONS AND ANSWERS

1/ Lack of convergence and compatibility between systems
Recommendation: overcome it in the preparatory stage, work it out with the host institution
2/ Differences in academic calendars, language requirements, and prerequisites among institutions hinder comparison and integration of academic programs
Recommendation: similarly, should be debated and solved at the LA negotiation process
3/ Recognition of credits and competencies:
Recommendation: Challenges in recognition of studied modules, credits transfer, and competency evaluation should proceed in accordance with the methodology that is known prior to the mobility
4/ Differences between European and non-European qualifications:
Recommendation: Recognition of non-European qualifications can be more challenging and time-consuming
Avoid potential pitfalls of the recognition process especially in the informal aspects of assessment:
Informal recommendations from teachers often play a crucial role in recognition, which can lead to bias.
Misrecognition of immigrants’ vocational experiences.
Validation practices may contribute to misrecognition of immigrants’ prior learning and experiences.

Degrading of knowledge:
Some immigrants with academic qualifications may only be given recognition at a lower educational level.
5/ How to overcome language barriers:
Newcomers may face challenges due to language differences, even if they have relevant vocational expertise
Recommendation: offer preparatory language courses for free in the before mobility period
and intercultural communication classes
6/ Costs and administrative burdens:
Students may face expenses for recognition procedures and additional administrative charges. These should be avoided by all means.
7/ Avoid lack of clear appeal processes:
In some cases, there may be no clear process for appealing rejected recognition decisions. Avoid inconsistent recognition practices such as cases when recognition depends on individual faculty councils, leading to potential inconsistencies.
8/ Mobility windows
Introduce mobility windows if possible already in the accreditation.
9/ Insufficient course information:
Lack of comprehensive and accessible course information makes it difficult for students to select appropriate courses and ensure credit recognition. Make the course catalogues of the ssending as well as receiving institution accessible online.
10/ Including special needs participants in the mobilities:
key points on how special needs participants in international mobilities should be treated and supported:

Equal treatment and access:
Special needs participants should receive equal treatment in all academic and service matters.
They should have full access to the same range of services and facilities available to local students and staff.

Additional support:
Erasmus+ offers additional financial support for students and staff with special needs to cover extra costs related to their participation. See Erasmus guide.
This can include funding for personal assistants, adapted accommodations, etc.

Individualized assistance:
Institutions should provide personalized support to special needs participants.
This may involve assigning dedicated staff members or offices to assist with practical matters.

Accessibility information:
Home institutions should provide accurate information on the accessibility levels of host institutions.
Only 21.6% of students with disabilities reported receiving such information, indicating a need for improvement.

Awareness of available support:
Better promotion of available support, like the Erasmus+ special needs grant, is needed. Only 15.1% of students with disabilities were aware of this grant.

Integration efforts:
Host institutions should make active efforts to integrate special needs participants, both academically and socially.
This can include organizing inclusive activities and providing necessary accommodations in classes.

Staff training:
Universities should provide training on accessibility and inclusion for staff members.

Collaboration between institutions:
Home and host institutions should collaborate closely to ensure proper support for special needs participants throughout their mobility.

Webinar 4 – QUESTIONS AND ANSWERS

1/ How shall we best formulate learning outcomes (LO)?
Here are some key guidelines for formulating effective learning outcomes:
a/ Use Clear, Specific Language
Learning outcomes should be written in clear, precise language that describes observable and measurable behaviors. Avoid vague or ambiguous terms.
For example: Instead of: “Students will understand calculus”
Use: “Students will be able to solve differential equations”
b/ Start with an Action Verb
Begin each learning outcome with an action verb that specifies the depth of learning expected
Use Bloom’s Taxonomy to select appropriate verbs for the desired cognitive level.
For example:
Remember/Understand: define, identify, describe
Apply/Analyze: calculate, compare, differentiate
Evaluate/Create: design, develop, formulate
Include Content, Context and Criteria
c/ A well-formulated learning outcome contains three key components
Action: The verb describing what the student will be able to do
Content: The specific subject matter or skills involved
Context: The situation or conditions under which the learning will be demonstrated
Criteria: The level of performance expected
d/ Use Student-Centered Language
Frame outcomes from the student’s perspective, using phrases like “Students will be able to…”. This clarifies expectations for learners.
Example:
By the end of the course, the student will (formulate it with active and action verbs) …. by (insert date and resources). In that way students can identify the long-terms goals, the tools and the process.

2/ What should be the limits LOs?
a/ Avoid Oversimplification
Learning outcomes can give a false impression of precision and simplicity, when in reality, learning is often complex and nuanced. There’s a risk of oversimplifying knowledge and skills, especially in disciplines with complex conceptual frameworks.
b/ Don’t Restrict Open-Ended Learning
LOs shouldn’t be so narrowly defined that they limit intellectual challenge or restrict open-ended academic inquiry. They should allow for exploratory and experimental teaching approaches.
c/ Be Wary of Overemphasis on Measurement
Focusing too heavily on measurable outcomes may cause educators to lose sight of valuable learning that is difficult to measure quantitatively. Not all important educational goals can be easily expressed as LOs.
d/ Consider Disciplinary Differences
The appropriateness and formulation of LOs can vary significantly between disciplines. A one-size-fits-all approach may not work across different fields of study.
e/ Avoid Excessive Detail
Having too many highly specific LOs can make a curriculum overly rigid and leave little room for flexibility or variation in teaching and learning
f/ The number of LOs should be manageable.
g/ Ensure Realistic Scope
LOs must be achievable within the timeframe of a course and assessable with available methods. They shouldn’t describe skills for future careers beyond the course.
h/ Allow for Unplanned Learning
Not all valuable learning can be predetermined and governed by Los. There should be space for unexpected learning outcomes to emerge.
i/ Don’t Neglect Implementation
Simply having well-written LOs doesn’t guarantee their effective use. Careful implementation and alignment with teaching activities and assessment is crucial. By being mindful of these limitations, educators can formulate more effective learning outcomes that enhance rather than constrain the educational process.

3/ How should staff be trained regarding LO?
To effectively train staff on learning outcomes (LOs):
a/ Provide Clear Explanations
Start by clearly explaining what learning outcomes are and why they are important. Staff should understand that Los define specific, measurable goals for student learning.
b/ Guide course design, teaching methods, and assessment – provide training and manuals
c/ Help align curriculum with broader program and institutional objectives
d/ Offer Practical Workshops
Conduct hands-on workshops where staff can practice writing and refining learning outcomes. Taxonomy
e/ Use Real Examples
Provide examples of well-written learning outcomes from various disciplines. Analyze these examples to demonstrate best practices and common pitfalls.
f/ Train staff on how to align learning outcomes with:
Course activities and materials
Assessment methods
Program-level outcomes
Institutional goals
This alignment ensures coherence in curriculum design and delivery.
g/ Teach Assessment Techniques
Instruct staff on how to assess whether learning outcomes have been achieved.
This includes:
Designing appropriate assessment tasks
Using rubrics and criteria
Collecting and analyzing evidence of student learning
h/ Encourage Collaboration
Promote collaborative approaches to developing and refining learning outcomes. This can involve:
i/ Peer review sessions
j/ Departmental discussions on program-level outcomes
k/ Cross-disciplinary sharing of best practices
l/ Provide Ongoing Support
Offer continuous support through:
Individual consultations
Follow-up workshops
Resources and guidelines
Regular review and feedback on LOs
Address Common Challenges

Prepare staff to handle common issues such as:
4/ Should there be a continuous training regarding LOs to the already hired staff?
a/ Adapting to Changes
As curricula, teaching methods, and educational goals evolve, staff need to stay updated on how to effectively use and formulate LOs. Continuous training ensures that all staff members are aligned with current best practices.
b/ Skill Enhancement
Regular training helps staff refine their skills in writing, implementing, and assessing LOs. This leads to more effective course design and improved student learning experiences.
c/ Maintaining Consistency
Ongoing training promotes a consistent approach to LOs across departments and programs, ensuring that all courses within an institution maintain a similar standard and structure.
d/ Benefits of Continuous LO Training
Improved Teaching Quality
As staff become more proficient in using LOs, the overall quality of teaching and course design is likely to improve.
Enhanced Student Experience
Well-crafted and consistently applied LOs can lead to clearer expectations for students and more focused learning experiences.
Institutional Alignment
Regular training helps ensure that all staff members are aligned with the institution’s educational goals and standards.

5/ How should Los be measured and monitored?
Learning outcomes (LOs) should be measured using a variety of methods to ensure comprehensive and accurate assessment.
A/ Use Direct and Indirect Measures
a/ Direct Measures
These assess actual student performance or behavior:
– Exams and quizzes
– Essays and research papers
– Presentations
– Projects and portfolios
– Lab reports
– Clinical evaluations

b/ Indirect Measures
These assess perceptions or reflections on learning:
– Surveys
– Reflective discussions
– Journals
– Self-assessments
– Alumni feedback

B/ Employ Quantitative and Qualitative Methods
a/ Quantitative Methods
– Rubrics with numeric scores
– Grading scales
– Standardized tests

b/ Qualitative Methods
– Narrative feedback
– Peer evaluations
– Observational assessments

C/ Measure LOs throughout the course, not just at the end.
This allows for:
– Tracking progress over time
– Identifying areas needing improvement
– Adjusting teaching methods as needed

D/ Utilize Technology
Leverage learning management systems and assessment software to:
– Streamline data collection
– Analyze results efficiently
– Generate reports on LO achievement

E/ Involve Multiple Assessors
When possible, have multiple instructors or external evaluators assess student work to enhance reliability and reduce bias.

F/ Consider Long-Term Impact
Assess the long-term achievement of LOs through:
– Follow-up surveys with graduates
– Employer feedback
– Professional certification rates

By implementing a comprehensive and varied approach to measuring learning outcomes, institutions can gain a more accurate picture of student achievement and use this information to continuously improve their educational programs.

6/ How should we implement the quality assurance of LOs?
a/ Establish a Systematic Review Process
b/ Implement a regular cycle for reviewing and refining LOs:
Annual reviews within departments
Comprehensive program-wide reviews every 3-5 years
Involve faculty, administrators, and external stakeholders
c/ Ensure that LOs are:
Reflected in course content and activities
Directly linked to assessment methods
Mapped to program and institutional goals
d/ Use SMART Criteria
Evaluate LOs against SMART criteria:
Specific
Measurable
Achievable
Relevant
Time-bound
e/ Implement Continuous Assessment
Use both direct and indirect measures to assess LO achievement:
Direct: exams, projects, presentations
Indirect: surveys, reflective discussions, alumni feedback
Collect data throughout the course, not just at the end

f/ Involve Multiple Stakeholders
g/ Engage various groups in the quality assurance process:
Faculty peers for review and feedback
Students for input on clarity and relevance
h/ Use assessment results to:
Identify areas for improvement
Modify course content or teaching methods
Update LOs as needed
Document these changes to demonstrate continuous improvement.

7/ How should we ensure the quality and feasibility of Los during the mobility and after the mobility?
A/ During Mobility
a/ Regular Check-ins
b/ Implement a system of regular check-ins with mobile students to:
c/ Assess progress towards LOs
d/ Identify any challenges or obstacles
e/ Provide support and guidance as needed
f/ Continuous Assessment
g/ Use formative assessment methods to track student progress:
Short quizzes or assignments
Reflective journals
Progress reports
Flexibility and Adaptation
h/ Be prepared to adjust LOs if necessary:
i/ Respond to unexpected learning opportunities
j/ Address any misalignments between expectations and reality
k/ Host Institution Collaboration
Work closely with the host institution to:
Ensure alignment of teaching methods with LOs
Address any cultural or academic differences that may impact LO achievement

B/ After Mobility
a/ Comprehensive Evaluation
Conduct a thorough assessment of LO achievement:
Use a mix of quantitative and qualitative measures
b/ Include self-assessment by students
c/ Gather feedback from host institution instructors
d/ Recognition and Integration
e/ Ensure that achieved LOs are:
Properly recognized and credited at the home institution
Integrated into the student’s overall academic record
Reflection and Application
f/ Encourage students to reflect on their learning:
g/ Organize debriefing sessions
h/ Assign reflective essays or presentations
i/ Discuss how to apply new knowledge and skills in their home context
j/ Long-term Impact Assessment
k/ Implement measures to evaluate the long-term impact of mobility on LOs:
l/ Follow-up surveys several months after return
m/ Track academic performance in subsequent courses
n/ Assess career outcomes related to mobility experience
Quality Assurance Measures:
a/ Feedback Loop
b/ Create a feedback mechanism to use insights from returned students to improve future mobility programs and LO formulation
c/ Data Analysis
Analyze data on LO achievement to identify trends, best practices, and areas for improvement in mobility programs
Engage various stakeholders in the quality assurance process:
Students
Faculty from both home and host institutions
Mobility program coordinators
Employers (for career-related LOs)

8/ LOs and Accreditation – what are the standards for LOs for study programme?
key standards for learning outcomes (LOs) in relation to accreditation of study programs:
a/ Alignment with Qualifications Frameworks
b/ LOs should be aligned with relevant qualifications frameworks:
European Qualifications Framework (EQF)
National qualifications frameworks
Discipline-specific frameworks (e.g., for medical education)
Specificity and Measurability
c/ LOs must be:
Specific and clearly described
Measurable and assessable
Achievable within the timeframe and resources of the program
Comprehensive Coverage
d/ LOs should encompass:
Knowledge, skills, and competencies
Different cognitive levels (e.g., using Bloom’s Taxonomy)
Professional values and behaviors
Relevance and Currency
e/ LOs need to:
Reflect current professional standards and practices
Address regional and national needs
Align with the institution’s mission and program aims
Inclusivity and Accessibility
f/ LOs should:
Be inclusive and based on universal design for learning
Incorporate diverse perspectives, including gender
Documentation and Transparency
g/ Programs must:
Clearly document and communicate LOs to students
Provide evidence of student achievement of LOs
Continuous Review and Improvement
There should be:
Regular review and updating of LOs
Use of assessment data to improve LOs and curriculum
Stakeholder Involvement
Development and review of LOs should involve:
Faculty
Students
External stakeholders (e.g., employers, professional bodies)
Integration with Curriculum and Assessment
g/ LOs must be:
Reflected in course content and activities
Directly linked to assessment methods
Mapped to program and institutional goals

Webinar 3 – QUESTIONS AND ANSWERS

1. Hello Prof. Jasmina! Thank you for your presentation! Can you please elaborate more on the balance
between knowledge, skills and competence?

It is not expected that each LO will precisely balance the elements of knowledge, skills, and competence. Each element should be included in the proper and relevant amount and follow the course objective. If the course is more practical, the focus will be more on skills and competence. If it is a theoretical course, it will focus more on knowledge. The point is to include all three elements.

2. A question about professional studies. Are there any guidelines more directed towards professional studies that could apply to courses within these study programmes in comparison to those offered by universities?

There should not be a difference between writing LOs in professional or academic studies. Knowledge, skills and competence are expected in both contexts. However, depending on the actual course, the LOs will probably have a stronger focus on the practical aspects of a course. If for example, the professional study programme is educating future elementary school teachers, the theoretical frameworks within a course will be different than in an academic theoretical course. The actual structure of the LOs will still follow the main ideas proposed in this webinar to use the vocabulary made available in Bloom’s taxonomy and Bloom’s revised taxonomy and cherry-pick the words that apply to the practical aspects of the specific course if that is what it is about.

3. The last slide was about skills ranking, from remember to create. Is there any rule that compares the
difficulties of those levels, e.g. the lowest level is 50 and the highest one is 100? We can use this rule in exam evaluation.

No, there is no rule. It would be difficult to quantify the amount of lower-order and higher-order thinking skills. The point is to use the progression suggested by the levels as a sort of direction.

4. Who evaluates your syllabi?

In Serbia, the reviewing committee appointed by the relevant accreditation body evaluates the syllabi when the higher education institution initiates the accreditation procedure by submitting the relevant documents.

5. Could you elaborate on which levels of knowledge are preferred to be used, for example, at the
Bachelor’s or Master’s level?

There is no rule regarding the specific individual levels. They all apply to all three levels, BA, MA and PhD. We can and should include them all as much as possible.

6. You talked about revised taxonomy and do the experts approve the revised taxonomy? The question
refers to the accreditation of the syllabi.
Both Bloom’s taxonomy and Bloom’s revised taxonomy are results of research based on empirical findings. We do not mention the taxonomies in our LOs or syllabi. We rely on the vocabulary suggested in them and the main notions proposed with lower-order and higher-order thinking skills. The accreditation committee does not ask about who or what the LOs are based upon. They want to see properly written syllabi, including concrete and specific LOs.