Lectures on: Mondays 13:30 – 15:30
Tutorials on: Mondays 15:30 – 16:30
Instructors: Yaniv David [yanivmd@technion.ac.il] , Aviv Shitrit, TA, [aviv.shitrit@campus.technion.ac.il]
Credits: 3 points
Prerequisites: 2340123 / 460209 (Operating Systems)
Course Objectives
- Understanding the operational architecture of agent-based systems: language models, external tools, memory, information sources, permissions, and runtime decision-making.
- Learning the attack surfaces unique to agentic systems, including direct and indirect prompt injection, information leakage, safety-mechanism bypasses, tool misuse, attacks on coding agents, and supply-chain poisoning in extension and plugin ecosystems.
- Acquiring the ability to perform threat modeling for an agent-based system: sensitive assets, trust boundaries, potential attackers, attack paths, and the security implications of agent actions.
- Learning secure-design principles and defenses, such as separating data from instructions, restricting capabilities, enforcing policies, controlling information flow, verifying sensitive actions, and monitoring with human intervention.
- Critically and empirically evaluating the robustness of defended agentic systems through current research, adaptive attacks, and analysis of success and failure rates.
Grade
- 10% for active participation in the lessons
- 30% 2-3 Homework assignments
- 60% Final Project
Homework
- During the semester 2-3 exercises will be published.
- Each with Wet and Dry parts.
- The submissions of the exercises are in singles!
- Submission of exercises is mandatory.
- AI Usage policy will be published for each assignment individually.
Final Project
- All the students will have to submit a final project in singles or groups.
- Groups will be approved based on project topic and scope by the staff.
- The final project will include research work based on the course topics.
Related Courses at the Technion
- Computer Security (02360490)
- Secure programming (02360491)
- Adversarial attacks on deep learning models and cyber security systems (02360207)