top of page

Reverse Engineering For Hacking (Beginner's Guide)

Reverse Engineering For Hacking (Beginner's Guide) | Black Hat HQ

Reverse Engineering For Hackers (Beginner's Guide)


Reverse engineering (RE) is analyzing software/binary to understand behavior, find vulnerabilities, extract secrets, or bypass protections. Essential for binary exploitation, malware analysis, and patch analysis. This is a guide on reverse engineering for hacking (Beginner's Guide).


Core Disciplines


Area

Focus

Tools

Static Analysis

Read code without executing

IDA Pro, Ghidra, radare2

Dynamic Analysis

Monitor behavior at runtime

x64dbg, GDB, strace

Malware Analysis

Understand malicious binaries

Cuckoo sandbox, CAPA

Firmware Analysis

Router/IoT reverse engineering

binwalk, Firmadyne


  1. Essential Tools (Kali)


# Install
sudo apt install radare2 gdb binutils strace ltrace strings

# Additional tools:
# - Ghidra (NSA's RE framework) → www.ghidra-sre.org
# - x64dbg (Windows) → Wine or dual-boot
# - IDA Free → hex-rays.com

Tool

Best For

Learning Curve

strings

Quick recon (pull readable text)

Zero

radare2

Command-line disassembly

Medium

Ghidra

Full decompilation to C

Medium

GDB

Linux binary debugging

Medium

strace/ltrace

Syscall/library call tracing

Low

objdump

Binary headers, sections, symbols

Low


  1. Quick-Wins (5-Minute Techniques)


Strings Analysis (Always First)


strings vulnerable.exe | grep -iE "password|admin|key|secret|flag|http|https"
strings -n 6 vulnerable.exe | head -50  # Minimum 6 chars

# Find embedded URLs or IPs
strings vulnerable.exe | grep -E "[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}"

# Look for encrypted/obfuscated data (high entropy)

Binary Metadata


file vulnerable.exe          # PE32, ELF, architecture
objdump -f vulnerable.exe    # Entry point, architecture
objdump -t vulnerable.exe    # Symbol table (if not stripped)
objdump -p vulnerable.exe    # PE header, imports/exports

Library Calls (Understand Behavior)


# Linux: trace all library calls
ltrace ./vulnerable_binary "test"
# Look for: strcmp, fopen, printf, system, execve

# Windows equivalent: API Monitor

  1. radare2 Beginner Workflow


# Open binary in read-only mode
r2 ./vulnerable_binary

# Analyze automatically
[0x00401000]> aaaa            # Full analysis
[0x00401000]> afl             # List all functions

# Look for main function
[0x00401000]> afl~main
[0x00401000]> s main          # Seek to main

# Disassemble
[0x00401150]> pdf             # Print disassembly of function
[0x00401150]> VV              # Visual control flow graph

# Search for strings
[0x00401150]> izz             # List all strings
[0x00401150]> izz~password    # Filter for "password"

# Find cross-references
[0x00401150]> axt str.password_compare  # Who references this string?

# Patching (change behavior)
[0x00401150]> s 0x00401200    # Seek to compare instruction
[0x00401200]> wa jmp 0x401300 # Write assembly (jump over check)

# Save modified binary
[0x00401200]> q
$ r2 -w vulnerable.exe        # Reopen writable

Quick Crackme Solution:


$ strings crackme | grep flag
$ r2 -qc "aaa; s main; VV" crackme  # Visual graph of main
# Look for comparison instructions → see hardcoded password
$ r2 -qc "aaa; is~password" crackme  # Find password symbol

  1. GDB Dynamic Analysis (Linux)


# Start debugging
gdb ./vulnerable_binary

# Set breakpoints
(gdb) break main
(gdb) break *0x401200         # Break at specific address
(gdb) break strcmp            # Break on string comparison

# Run with arguments
(gdb) run AAAA               # Try input "AAAA"
(gdb) run $(python3 -c "print('A'*100)")  # Buffer overflow test

# Examine registers
(gdb) info registers
(gdb) x/10x $rsp             # Look at stack

# Single step
(gdb) stepi                  # Step one instruction
(gdb) nexti                  # Step over calls

# Examine memory
(gdb) x/s 0x7fffffffe000     # Print string at address
(gdb) x/10gx $rsp            # 8-byte chunks on stack

# Continue execution
(gdb) continue
(gdb) quit

  1. Practical Reverse Engineering Techniques


Finding Hardcoded Credentials


# Static
strings binary | grep -E "\.com|\.net|@|password|user|admin"

# Dynamic (monitor strcmp calls)
ltrace ./binary fake_password 2>&1 | grep strcmp

Bypassing License/Registration Checks


# Find check point
strings binary | grep -iE "license|registered|trial|expired"

# In radare2
[0x00401150]> axt str.You_are_not_registered
# Jump to cross-reference → patch JZ (jump if zero) to JNZ or NOP

Extracting Embedded Data


# Extract resources
binwalk -e firmware.bin              # Extract files
dd if=binary bs=1 skip=OFFSET of=data.bin  # Manual extraction

# Reconstruct from memory dump
objdump -s -j .rodata binary         # Read-only data section

Simple Crackme Pattern (Password Check)


$ r2 -qc "aaa; izz~flag" crackme
# Found: "Correct! Flag: XXXX"

$ r2 -qc "aaa; pdf @ main" crackme
# See: strcmp(input, hardcoded_string)
# Solution: run binary with hardcoded string as input

  1. Ghidra Beginner Guide


Ghidra decompiles binary to pseudo-C (much easier than raw assembly).


Quick Start:


# Launch
ghidraRun

# Create project → Import binary → Analyze

# Key windows:
# - Symbol Tree: functions, strings, imports
# - Listing: disassembly
# - Decompiler: pseudo-C code (gold mine)
# - Data Type Manager: structs

Typical Ghidra Workflow:


  1. Import binary → auto-analyze.

  2. Symbol Tree → Functions → main (double-click).

  3. Decompiler window shows readable C-like code.

  4. Look for strcmpmemcmpif statements checking input.

  5. Search strings → find references → understand logic.

  6. Patch: right-click instruction → "Patch Instruction" → change JZ to JNZ.


  1. Common RE Patterns


Pattern

Assembly

What It Means

strcmp

call strcmp; test eax,eax; jne fail

Password validation

Buffer overflow

mov rcx, [rbp-0x100]; call gets

No bounds check

Format string

mov rcx, format; mov rdx, input; call printf

Printf vuln

Integer overflow

add eax, ebx; jo overflow_handler

Arithmetic wrap

Anti-debug

call ptrace; test eax,eax; je continue

Debugger detection


  1. Complete Beginner RE Workflow


Objective: Find hardcoded password in Linux binary.


# Step 1: Recon
$ file target_binary
target_binary: ELF 64-bit LSB executable, x86-64
$ strings target_binary | head -20
[Found: "Enter password:", "Access denied", "Flag{"]

# Step 2: Dynamic (quick)
$ ltrace ./target_binary test123
strcmp("test123", "s3cr3t_p@ss")  = -1  # HARDCODED PASSWORD FOUND

# Step 3: Static (deeper)
$ r2 -qc "aaa; pdf @ main; izz~pass" target_binary
[See disassembly → strcmp with hardcoded string]

# Step 4: Verify
$ ./target_binary s3cr3t_p@ss
"Access granted! Flag: FLAG{reverse_engineering_101}"

  1. Windows Binary Analysis


# Check imports (tells you what it does)
objdump -p malware.exe | grep -i "DLL Name"

# Common suspicious imports:
# - kernel32.dll: CreateProcess, WriteProcessMemory
# - ws2_32.dll: socket, connect (network)
# - advapi32.dll: CryptEncrypt (ransomware)
# - wininet.dll: InternetOpenUrl (C2 download)

# Extract resources
wrestool -x malware.exe -o extracted/

  1. Anti-Reverse Engineering & Bypass


Protection

Bypass

Stripped symbols

Heuristic function naming (Ghidra's)

Obfuscated strings

Dynamic analysis → dump memory at runtime

Anti-debug (ptrace)

Patch call to ptrace → NOP

Packed (UPX)

upx -d binary.exe or manual unpack

CRC checks

Patch check → jump over


Unpacking UPX:


upx -d packed.exe -o unpacked.exe
# Or manually dump from memory: GDB → dump memory

  1. Pentest Use Cases


Scenario

RE Technique

Tool

Test custom authentication

Find hardcoded keys/strcmp

radare2/Ghidra

Analyze dropped malware

Understand behavior

CAPA strings dynamic

Find buffer overflows

Look for gets, strcpy, scanf

radare2 pattern

Extract firmware secrets

binwalk → strings

binwalk


  1. Lab Practice


# Download crackmes:
git clone https://github.com/angr/angr-doc
git clone https://github.com/malware-unicorn/reverse-engineering-workshop

# Try MicroCorruption (embedded RE): microcorruption.com
# Try Crackmes.one (beginner level)

Quick Lab:

# Create test binary
echo "int main(){char b[16]; gets(b); printf(b); return 0;}" > vuln.c
gcc -o vuln vuln.c -fno-stack-protector -no-pie
strings vuln  # See gets + printf = format string + buffer overflow vulns
r2 -qc "aaa; afl~gets; pdf @ sym.imp.gets" vuln

Enroll In Online Cybersecurity & Hacking Classes/Courses | Black Hat HQ

Comments


bottom of page