Class Central is learner-supported. When you buy through links on our site, we may earn an affiliate commission.

YouTube

Architecture 2001 - Intel x86-64 OS Internals

OpenSecurityTraining2 via YouTube

Overview

Coursera Flash Sale
40% Off Coursera Plus for 3 Months!
Grab it
Explore the intricate details of Intel x86-64 processor architecture and operating system internals through this comprehensive 6-hour 54-minute course. Delve into fundamental concepts starting with processor execution modes, Model Specific Registers, and the CPUID instruction before progressing to advanced topics including privilege rings, segmentation, and segment registers. Master the Global Descriptor Table (GDT) and Local Descriptor Table (LDT) through hands-on laboratories that demonstrate real-world implementation. Examine interrupt handling mechanisms, including the differences between interrupts and exceptions, software interrupt instructions, Task State Segments (TSS), and the Interrupt Descriptor Table (IDT). Investigate system call implementations, comparing syscall versus sysenter instructions and their 32-bit versus 64-bit compatibility, while exploring syscall-adjacent technologies like swapgs and register base operations. Understand memory management through detailed coverage of paging mechanisms, including 4-level and 5-level paging structures, page table entries from CR3 to PTE level, canonical addresses, page faults, and Translation Lookaside Buffer (TLB) operations. Learn about security features such as the NX/XD bit for non-executable memory protection and exploit mitigation techniques including SMEP and SMAP. Practice debugging techniques using both software and hardware breakpoints, debug registers, and single-stepping mechanisms. Conclude with port I/O operations including CMOS interaction and VMware-specific implementations. Each module includes practical laboratories and real-world examples to reinforce theoretical concepts, making complex processor architecture accessible through hands-on experimentation and detailed analysis of actual system behavior.

Syllabus

Class Intro - Architecture 2001: x86-64 OS Internals
Warm Up: CPUID Instruction
Intel Processor Execution Modes
Model Specific Registers
Privilege Rings & Segmentation - Privilege Rings Start
Privilege Rings - Segmentation & Segment Registers 1
Privilege Rings - Segmentation & Segment Registers 2 - Labs: U_SegRegs & K_SegRegs
Privilege Rings - Segmentation & Segment Registers 3 - Optional: 32-bit Throwback
Global Descriptor Table (GDT) & Local Descriptor Table (LDT) 1 - Global Descriptor Table Register
Global Descriptor Table (GDT) & Local Descriptor Table (LDT) 2 - Lab: Look at the GDTR
Global Descriptor Table (GDT) & Local Descriptor Table (LDT) 3 - Local Descriptor Table Register
Global Descriptor Table (GDT) & Local Descriptor Table (LDT) 4 - Lab: Look at the LDTR
Global Descriptor Table (GDT) & Local Descriptor Table (LDT) 5 - GDTR & LDTR: What Did We Learn?
Privilege Rings - Segment Descriptors 1 - Segment Descriptors
Privilege Rings - Segment Descriptors 2 - Lab: Hand-parsing GDT Segment Descriptors & Using SwishDbg
Privilege Rings - Segment Descriptors 3 - What Did We Learn?
Privilege Rings & Segmentation - Privilege Rings End
Privilege Rings & Segmentation - Call Gates
Privilege Rings & Segmentation - A Return To Read(ing) The Fun Manual!
Privilege Rings & Segmentation - Implicit and Explicit Use of Segmentation
Privilege Rings & Segmentation - Conclusion
Interrupts - Interrupts vs. Exceptions
Interrupts - Software Interrupt Instructions
Interrupts - Tasks and the Task State Segment (TSS)
Interrupts - Tasks and the Task State Segment (TSS) - Lab: Examine a 64-bit TSS
Interrupts - Interrupt Descriptor Table (IDT)
Interrupts - Interrupt Descriptors
Interrupts - Interrupt Descriptors - Labs: Pearly Interrupt Gates 1 & 2
Interrupts - Interrupt Descriptors - What Did We Learn?
Interrupts - Interrupt Masking
Interrupts - Red Pill and Virtualization Detection
Interrupts - Conclusion
System Calls - System Calls Design Background
System Calls - syscall vs sysenter 32-bit vs. 64-bit Instruction Compatibility
System Calls - syscall/sysret: x86-64's Preferred System Call Instructions
System Calls - Visualizing the syscall MSR Usage
System Calls - Lab: Syscall MSR Investigation
System Calls - Optional: sysenter/sysexit: x86-32's preferred system call instructions
System Calls - Syscall-adjacent Tech (swapgs, {rd,wr}{fs,gs}base)
System Calls - Conclusion
Read the TimeStamp Counter (RDTSC) Assembly Instruction
Read the TimeStamp Counter (RDTSC) - Labs: U_Guestimate & U_NavelGaze
Paging - Paging Introduction
Paging - Paging and the Control Registers
Paging - Paging and the Control Registers - Lab: Read the Control Registers
Paging - Paging and the Control Registers - What Did We Learn?
Paging - Page Tables - MAXPHYADDR
Paging - Page Tables - 32 bit Linear to 32 bit Physical, 4KB Pages
Paging - Page Tables - 32 bit Linear to 32 bit Physical, 4MB Pages
Paging - Page Tables - 32 bit Linear to 40 bit Physical, Physical Address Extensions (PAE)
Paging - Page Tables - 48 bit Linear to 52 bit Physical, 4-level Paging, 4KB, 2MB, 1GB Pages
Paging - Page Tables - (Optional) 57 bit Linear to 52 bit Physical, 5-level Paging
Paging - Page Table Entries - CR3
Paging - Page Table Entries - CR3 - Lab: Page Table Walkabout 1
Paging - Page Table Entries - PML4E
Paging - Page Table Entries - PML4E - Lab: Page Table Walkabout 2
Paging - Page Table Entries - PML4E - Lab Example: Page Table Walkabout 2
Paging - Page Table Entries - Exploit Mitigation Aside: XD, SMEP, SMAP
Paging - Page Table Entries - PDPTE
Paging - Page Table Entries - PDPTE - Lab: Page Table Walkabout 3
Paging - Page Table Entries - PDPTE - Lab Example: Page Table Walkabout 3
Paging - Page Table Entries - PDE
Paging - Page Table Entries - PDE - Lab: Page Table Walkabout 4
Paging - Page Table Entries - PDE - Lab Example: Page Table Walkabout 4
Paging - Page Table Entries - PTE
Paging - Page Table Entries - PTE - Lab: Page Table Walkabout 5
Paging - Page Table Entries - PTE - Lab Example: Page Table Walkabout 5
Paging - Canonical Addresses
Paging - Page Faults
Paging - Page Faults - Lab: Seeing What the Page Fault Handler Sees
Paging - Page Faults - Misc Points
Paging - Translation Lookaside Buffer (TLB)
Paging - Translation Lookaside Buffer (TLB) - Shadow Walker Rootkit
Paging - Non-executable Memory (NX/XD bit)
Paging - Non-executable Memory (NX/XD bit) - Lab: Checking the kernel's stack execute permissions
Paging - Non-executable Memory - Lab2: Checking a userspace process' stack execute permissions
Paging - Conclusion
Debugging - Software breakpoints
Debugging - Hardware Breakpoints - Hardware Debug Registers
Debugging - Hardware Breakpoints - Lab: Checking Debug Registers
Debugging - Hardware Breakpoints - Lab Walkthrough: Checking Debug Registers in WinDbg
Debugging - Hardware Breakpoints - Resume Flag (RF)
Debugging - Hardware Breakpoints - Trap Flag (TF) and Single-Stepping
Port I/O - Port I/O
Port I/O - Lab: K_VMWare_PortIO
Port I/O - CMOS Background
Port I/O - Lab: K_CMOS_PortIO
Conclusion - Architecture 2001: x86-64 OS Internals

Taught by

OpenSecurityTraining2

Reviews

Start your review of Architecture 2001 - Intel x86-64 OS Internals

Never Stop Learning.

Get personalized course recommendations, track subjects and courses with reminders, and more.

Someone learning on their laptop while sitting on the floor.