Logo
What is System Calls

What is System Calls

SH∆FIQ∆IM∆N SH∆FIQ∆IM∆N
May 23, 2025
5 min read
index

Opening LOL

Well, this is my first ever blog post that’s not a writeup. Why am I even writing this blog post? 🤷 You might be curious, right? …no? Well, I’m going to tell you anyway.

It all started when my best friend/colleague gave me a task: dumping LSASS without using the good ol’ Mimikatz. So, we did some digging on this beautiful web of lies I mean, the internet.
We found a couple of interesting things. What amazed me most was how many tools used techniques I never thought I’d ever learn or even use. That exposure got me really into it. I started enjoying the whole process of evasion, stealth, and all that cool stuff.

Enough chit-chat—let’s get started, shall we? 😅

Important

Everything I’ve learned is based on blog posts and writeups from people who did some amazing work. Props to them!
All the resources are listed below.
YES!.. This is basically a blog about me trying my best to understand Syscalls.

User vs Kernel – fight!

But before we jump straight into syscalls, we need to understand how Windows operates. The Windows Operating System is divided into two modes: user mode and kernel mode.

But why do these modes even exist in the first place? 🧐

The purpose is to protect critical OS components “kernel mode” from being accessed or modified by regular applications “user mode”. This separation ensures stability and security.

Example

A user application like Chrome runs in user mode, while OS components like system services and drivers operate in kernel mode.

So, what is the kernel anyway?

In layman’s terms, the kernel is the heart of the Operating System. It’s where real execution happens on the processor. It has access to all system memory and all CPU instructions.

My precious!

But wait! Don’t processors have different architectures?

How does the system differentiate between modes? Glad you asked 👀.
While processors do come in different architectures (like x86 and x64), they use something called privilege levels, or ring levels, to enforce security and access control.

These rings range from the least privileged (Ring 3) to the most privileged (Ring 0):

ring levels

Ring Levels

  • Ring 3
    Also known as user mode, where all regular applications (like browsers, text editors, etc.) run. They have limited access to system resources.

  • Ring 2 & 1
    Ring 2: Typically handles device drivers and storage operations (e.g., HDDs, SSDs).
    Ring 1: Used for things like managing communication between the OS and hardware.

  • Ring 0
    Also known as kernel mode. Everything running here has direct access to the CPU and system memory. This includes the OS kernel, drivers, and core system processes.

Let’s burn this bridge down shall we? 🔥

However, programs also need to interact with the kernel to run properly, for tasks like memory allocation, file creation, etc.

So, how does a user-mode program communicate with the kernel?

Well, it needs a bridge to travel from user mode to kernel mode. This bridge is made up of three steps:

bridge

Build a bridge and get over it

The Win32 API is the main API used in user mode. It’s a collection of DLLs (Dynamic-Link Libraries), such as Kernel32.dll, User32.dll, etc. What’s the deal with these DLCs DLLs? Basically, they contain functions that applications can call.
I summon thee! 🔮

Now, let’s talk about the Native API, also known as ntdll.dll. This one’s arguably the most iconic DLL on the list (in my opinion). Why? Because it sits closest to the kernel. It’s the last stop in user mode before crossing over to kernel mode. It makes transitioning between these modes smooth and efficient.

Now!!! The grand reveal!!! SYSCALL (give it a round of applause 👏) LOL.

Well, the transition from user mode to kernel mode is done via the syscall instruction, which is implemented inside ntdll.dll.

Syscall third-wheeling

Syscall third-wheeling

So, the full flow looks something like this:

Application flowpath

Application flowpath

We can observe this transition happening in any application by using Process Monitor. For this demonstration, we’ll take a look at the good ol’ Notepad.exe.

Transition user to kernel

Transition user to kernel

Recall

Based on the image above, you can see the flow we talked about earlier:
KernelBase.dll (Win32 API) → ntdll.dll (Native API) → ntoskrnl.exe (Kernel).

Also, check out the functions being used:
CreateFileW (user mode) → NtCreateFile (user mode) → NtCreateFile (kernel mode).

Hold up, wait a minute

Did you notice that NtCreateFile is called twice, once in ntdll.dll and again in ntoskrnl.exe. Why’s that?
It’s because ntdll.dll contains a stub or wrapper for the function that’s actually implemented in the kernel “ntoskrnl.exe”. Think of it like this:

  • ntdll.dll -> Gateway
  • ntoskrnl.exe -> Real deal baby!!
Tip

Imagine it as a “Kernel API”, where the syscall is just passing the request from userland to the real implementation inside the kernel.

Understood? Good! 😅🫠

Skeletor - until we meet again

As Promised:
Here Are the Resources 📑 Everything I’ve explored and shared here came from reading and experimenting based on the incredible work of others. If you’re curious or want to go deeper, definitely check these out: