Reverse engineering tutorial and challenge

Written on September 6, 2016 by Gabor Pek

So here we are again with your next avatao Tuesday challenge. Today, we are delving a bit into reverse engineering by providing a small tutorial and a challenge to solve.

A decent definition for reverse engineering comes from Eldad Eilam from his Reversing: Secrets of Reverse Engineering book: “In the software world reverse engineering boils down to taking an existing program for which source-code or proper documentation is not available and attempting to recover details regarding its’ design and implementation.”

You can easily grasp the idea behind this definition if you write a simple C program, compile and disassemble it. For simplicity, we are going to create a simple Linux ELF binary with GCC

So here is your first source code:

#include <stdio.h>

int main()
{
    printf("Hello avatao Tuesday\n");
    return 0;
}

Let’s create a 32-bit binary from this source code:

gcc -m32 -o re_challenge re_challenge.c

If you prefer 64 bit simply use -m64

gcc -m64 -o re_challenge re_challenge.c

There are various disassemblers available online in demo version (e.g., IDA, Binary Ninja) or entirely free (e.g., radare2). In this tutorial, we are going to use IDA to dissect our 32-bit binary. If you simply open the binary in IDA you will see something similar:

re_challenge

The compiled binary contains instructions that can be executed by the CPU directly. The language which makes these machine instructions readable for humans is called Assembly. That is what we generally work with while reverse engineering binaries.

Every architecture comes with an instruction set which is typically documented by vendors. In our case, you can find the details in the Intel 64 and IA-32 Architectures Software Developer Manuals.

In short, the code above first prepares space for the stack frame by aligning and moving the stack pointer by means of the esp CPU register. After that the code pushes the address of the Hello avatao Tuesday string to the stack and calls the puts function. Due to the cdecl calling convention puts will search its function argument on the top of the stack where the address of our string is located.

For more information about the topic we suggest to read Dennis Yurichev’s guide.

All right. I am sure that after this small introduction you can solve the second avatao Tuesday challenge on reverse engineering!