Hooking


In computer programming, the term hooking covers a range of techniques used to alter or augment the behaviour of an operating system, of applications, or of other software components by intercepting function calls or messages or events passed between software components. Code that handles such intercepted function calls, events or messages is called a hook.
Hooking is used for many purposes, including debugging and extending functionality.
Examples might include intercepting keyboard or mouse event messages before they reach an application, or intercepting operating system calls in order to monitor behavior or modify the function of an application or other component. It is also widely used in benchmarking programs, for example frame rate measuring in 3D games, where the output and input is done through hooking.
Hooking can also be used by malicious code. For example, rootkits, pieces of software that try to make themselves invisible by faking the output of API calls that would otherwise reveal their existence, often use hooking techniques.

Methods

Typically hooks are inserted while software is already running, but hooking is a tactic that can also be employed prior to the application being started. Both these techniques are described in greater detail below.

Runtime modification

Operating systems and software may provide the means to easily insert event hooks at runtime. It is available provided that the process inserting the hook is granted enough permission to do so. Microsoft Windows for example, allows you to insert hooks that can be used to process or modify system events and application events for dialogs, scrollbars, and menus as well as other items. It also allows a hook to insert, remove, process or modify keyboard and mouse events. Linux provides another example where hooks can be used in a similar manner to process network events within the kernel through NetFilter.
When such functionality is not provided, a special form of hooking employs intercepting the library function calls made by a process. Function hooking is implemented by changing the very first few code instructions of the target function to jump to an injected code. Alternatively on systems using the shared library concept, the interrupt vector table or the import descriptor table can be modified in memory. Essentially these tactics employ the same ideas as those of source modification, but instead altering instructions and structures located in the memory of a process once it is already running.

Sample code

Virtual method table hooking

Whenever a class defines/inherits a virtual function, compilers add a hidden member variable to the class which points to a virtual method table. Most compilers place the hidden VMT pointer at the first 4 bytes of every instance of the class. A VMT is basically an array of pointers to all the virtual functions that instances of the class may call. At runtime these pointers are set to point to the right functions, because at compile time, it is not yet known if the base function is to be called or if an overridden version of the function from a derived class is to be called. Therefore, virtual functions can be hooked by replacing the pointers to them within any VMT that they appear. The code below shows an example of a typical VMT hook in Microsoft Windows, written in C++.

  1. include
  2. include "windows.h"
using namespace std;
class VirtualClass
using VirtualFn1_t = void;
VirtualFn1_t orig_VirtualFn1;
void __fastcall hkVirtualFn1 //This is our hook function which we will cause the program to call instead of the original VirtualFn1 function after hooking is done.
int main

It is important to note that all virtual functions must be class member functions, and all class member functions are called with the __thiscall calling convention. The __thiscall calling convention passes a pointer to the calling class instance via the ECX register. Therefore, in order for a hook function to properly intercept the "this" pointer that is passed and take it as an argument, it must look into the ECX register. In the above example, this is done by setting the hook function to use the __fastcall calling convention, which causes the hook function to look into the ECX register for one of its arguments.
Also note that, in the above example, the hook function is not a member function itself so it cannot use the __thiscall calling convention. __fastcall has to be used instead because it is the only other calling convention that looks into the ECX register for an argument.

C# keyboard event hook

The following example will hook into keyboard events in Microsoft Windows using the Microsoft.NET Framework.

using System.Runtime.InteropServices;
namespace Hooks

API/function hooking/interception using JMP instruction aka splicing

The following source code is an example of an API/function hooking method which hooks by overwriting the first six bytes of a destination function with a JMP instruction to a new function. The code is compiled into a DLL file then loaded into the target process using any method of DLL injection. Using a backup of the original function one might then restore the first six bytes again so the call will not be interrupted. In this example the win32 API function MessageBoxW is hooked.

/*
This idea is based on chrom-lib approach, Distributed under GNU LGPL License.
Source chrom-lib: https://github.com/linuxexp/chrom-lib
Copyright 2011 Raja Jamwal
  • /
  1. include
  2. define SIZE 6
typedef int ; // Messagebox prototype
int WINAPI MyMessageBoxW; // Our detour
void BeginRedirect;
pMessageBoxW pOrigMBAddress = NULL; // address of original
BYTE oldBytes = ; // backup
BYTE JMP = ; // 6 byte JMP instruction
DWORD oldProtect, myProtect = PAGE_EXECUTE_READWRITE;
INT APIENTRY DllMain
void BeginRedirect
int WINAPI MyMessageBoxW

Netfilter hook

This example shows how to use hooking to alter network traffic in the Linux kernel using Netfilter.

  1. include
  2. include
  3. include
  4. include
  5. include
  6. include
  7. include
  8. include
/* Port we want to drop packets on */
static const uint16_t port = 25;
/* This is the hook function itself */
static unsigned int hook_func)
/* Used to register our hook function */
static struct nf_hook_ops nfho = ;
static __init int my_init
static __exit void my_exit
module_init;
module_exit;

Internal IAT hooking

The following code demonstrates how to hook functions that are imported from another module.
This can be used to hook functions in a different process from the calling process. For this the code must be compiled into a DLL file then loaded into the target process using any method of DLL injection.
The advantage of this method is that it is less detectable by antivirus software and/or anti-cheat software, one might make this into an external hook that doesn't make use of any malicious calls.
The Portable Executable header contains the Import Address Table, which can be manipulated as shown in the source below. The source below runs under Microsoft Windows.

  1. include
typedef int ; //This is the 'type' of the MessageBoxA call.
pMessageBoxA RealMessageBoxA; //This will store a pointer to the original function.
void DetourIATptr;
int __stdcall NewMessageBoxA
int main
void **IATfind
void DetourIATptr

Windows